Electricity Bench

Grades over time

Every subject is re-run on a monthly cadence, and each run is kept. These are the trajectories that history builds: capability and latency per suite family, one line per agent.

Grades are comparable only within a suite version. When a suite changes, the task set changes, so the versions get a line each rather than one line sloping across the change - a v1 grade and a v2 grade are two different measurements, not two points on one trend. A suite change runs both versions for a cycle or two before the old one is retired, so the lines overlap in time and the superseded version is dashed. Plan cost is not charted here: it is a share of each agent's own subscription, so it is shown per agent on its scorecard instead of ranked across agents.

structured extraction

Suite versions in this history: v1 (from Jul 13, 2026), v2 (from Jul 13, 2026).

Capability

0%50%100%Jul 13Jul 13Jul 16v2 startsClaude Sonnet 5 (CLI) · v1 (superseded content) · Jul 13, 2026 · 100% · grade AClaude Sonnet 5 (CLI) · v2 · Jul 13, 2026 · 100% · grade AClaude Sonnet 5 (CLI) · v1 · Jul 16, 2026 · 100% · grade A0%50%100%Jul 13Jul 13Jul 16v2 startsClaude Sonnet 5 (CLI) · v1 (superseded content) · Jul 13, 2026 · 100% · grade AClaude Sonnet 5 (CLI) · v2 · Jul 13, 2026 · 100% · grade AClaude Sonnet 5 (CLI) · v1 · Jul 16, 2026 · 100% · grade A
Claude Sonnet 5 (CLI) · v1 (superseded content)Claude Sonnet 5 (CLI) · v2Claude Sonnet 5 (CLI) · v1dashed = a superseded question set: an older suite version still running alongside the current one, or content a version no longer holdsa new suite version starts - grades compare only within a version

Median latency

3715 ms4839 ms5964 msJul 13Jul 13Jul 16v2 startsClaude Sonnet 5 (CLI) · v1 (superseded content) · Jul 13, 2026 · 4397 ms · grade AClaude Sonnet 5 (CLI) · v2 · Jul 13, 2026 · 3974 ms · grade AClaude Sonnet 5 (CLI) · v1 · Jul 16, 2026 · 5704 ms · grade A3715 ms4839 ms5964 msJul 13Jul 13Jul 16v2 startsClaude Sonnet 5 (CLI) · v1 (superseded content) · Jul 13, 2026 · 4397 ms · grade AClaude Sonnet 5 (CLI) · v2 · Jul 13, 2026 · 3974 ms · grade AClaude Sonnet 5 (CLI) · v1 · Jul 16, 2026 · 5704 ms · grade A
Claude Sonnet 5 (CLI) · v1 (superseded content)Claude Sonnet 5 (CLI) · v2Claude Sonnet 5 (CLI) · v1dashed = a superseded question set: an older suite version still running alongside the current one, or content a version no longer holdsa new suite version starts - grades compare only within a version

Every run

RunAgentSuiteGradeCapabilityLatency
Jul 16, 2026Claude Sonnet 5 (CLI)v1A current100%5704 ms
Jul 13, 2026Claude Sonnet 5 (CLI)v2A current100%3974 ms
Jul 13, 2026Claude Sonnet 5 (CLI)v1 (superseded content)A100%4397 ms