Electricity Bench

Grades over time

Overall leads: each agent's score across every suite it has been graded on, one point per grading week. Switch to a single suite for its own line. Filter by provider or agent, click a point for the breakdown.

Overall is the capability mean across an agent's graded suites - the same rule the leaderboard's overall grade uses - with each suite contributing its latest reading, so a week that re-ran one suite moves the line by that suite's share of the mean rather than replacing it. The week a new suite joined the average is ruled off on the chart. Grades compare only within a suite version. Plan usage is each run's share of that agent's own subscription over one week, as the provider reports it - every agent is read on the same window, whatever window its provider declares, because the weekly limit is what these plans are sold on. The usage tab plots those shares side by side, but a percent of one plan is not a percent of another, so read it per agent, not as a ranking. The chart does not mark provider announcements; the dates of every plan and limit change, with the usage measured either side of each, live on the plan change log.

View

Real-world issues · Spec planning · Vibe coding · 13 agents · every suite counted, each contributing its latest grade

20%30%40%50%60%70%80%90%2026-W322026-W342026-W362026-W382026-W40+ vibe-coding+ spec-planningThe average counted a different set of suites either side of this segmentThe average counted a different set of suites either side of this segmentgemini-3.1-pro · 2026-W32 · Aug 5, 2026 · 47% · Dgemini-3.1-pro · 2026-W33 · Aug 11, 2026 · 34% · Fgemini-3.1-pro · 2026-W34 · Aug 17, 2026 · 34% · Fgemini-3.1-pro · 2026-W35 · Aug 24, 2026 · 38% · D-gemini-3.1-pro · 2026-W36 · Aug 31, 2026 · 35% · Fgemini-3.1-pro · 2026-W37 · Sep 7, 2026 · 28% · Fgemini-3.1-pro · 2026-W38 · Sep 14, 2026 · 28% · Fgemini-3.1-pro · 2026-W39 · Sep 21, 2026 · 35% · Fgemini-3.1-pro · 2026-W40 · Sep 28, 2026 · 34% · FThe average counted a different set of suites either side of this segmentThe average counted a different set of suites either side of this segmentNew registration - model gemini-3.7-flash -> gemini-3.8-flash, same harness, plan and settingsgemini-3.6-flash · 2026-W32 · Aug 5, 2026 · 60% · C-gemini-3.7-flash · 2026-W33 · Aug 14, 2026 · 72% · C+gemini-3.7-flash · 2026-W34 · Aug 18, 2026 · 57% · C-gemini-3.7-flash · 2026-W35 · Aug 24, 2026 · 55% · C-gemini-3.8-flash · 2026-W36 · Sep 3, 2026 · 48% · Dgemini-3.8-flash · 2026-W37 · Sep 7, 2026 · 59% · C-gemini-3.8-flash · 2026-W38 · Sep 14, 2026 · 53% · D+gemini-3.8-flash · 2026-W39 · Sep 21, 2026 · 44% · Dgemini-3.8-flash · 2026-W40 · Sep 28, 2026 · 56% · C-The average counted a different set of suites either side of this segmentThe average counted a different set of suites either side of this segmentNew registration - model claude-fable-5 -> claude-fable-5-1, same harness, plan and settingsclaude-fable-5 · 2026-W32 · Aug 5, 2026 · 60% · C-claude-fable-5 · 2026-W33 · Aug 11, 2026 · 72% · C+claude-fable-5 · 2026-W34 · Aug 18, 2026 · 64% · Cclaude-fable-5 · 2026-W35 · Aug 24, 2026 · 68% · Cclaude-fable-5-1 · 2026-W36 · Sep 1, 2026 · 77% · B-claude-fable-5-1 · 2026-W37 · Sep 7, 2026 · 71% · C+claude-fable-5-1 · 2026-W38 · Sep 14, 2026 · 67% · Cclaude-fable-5-1 · 2026-W39 · Sep 21, 2026 · 61% · C-claude-fable-5-1 · 2026-W40 · Sep 28, 2026 · 77% · B-The average counted a different set of suites either side of this segmentThe average counted a different set of suites either side of this segmentclaude-haiku-4-5 · 2026-W32 · Aug 4, 2026 · 30% · Fclaude-haiku-4-5 · 2026-W33 · Aug 11, 2026 · 26% · Fclaude-haiku-4-5 · 2026-W34 · Aug 18, 2026 · 37% · D-claude-haiku-4-5 · 2026-W35 · Aug 24, 2026 · 26% · Fclaude-haiku-4-5 · 2026-W36 · Aug 31, 2026 · 42% · Dclaude-haiku-4-5 · 2026-W37 · Sep 7, 2026 · 31% · Fclaude-haiku-4-5 · 2026-W38 · Sep 14, 2026 · 30% · Fclaude-haiku-4-5 · 2026-W39 · Sep 21, 2026 · 31% · Fclaude-haiku-4-5 · 2026-W40 · Sep 28, 2026 · 26% · FThe average counted a different set of suites either side of this segmentThe average counted a different set of suites either side of this segmentclaude-opus-5 · 2026-W32 · Aug 5, 2026 · 60% · C-claude-opus-5 · 2026-W33 · Aug 11, 2026 · 50% · D+claude-opus-5 · 2026-W34 · Aug 18, 2026 · 64% · Cclaude-opus-5 · 2026-W35 · Aug 24, 2026 · 60% · C-claude-opus-5 · 2026-W36 · Aug 31, 2026 · 62% · Cclaude-opus-5 · 2026-W37 · Sep 7, 2026 · 63% · Cclaude-opus-5 · 2026-W38 · Sep 14, 2026 · 66% · Cclaude-opus-5-5 · 2026-W39 · Sep 23, 2026 · 71% · C+claude-opus-5-5 · 2026-W40 · Sep 28, 2026 · 79% · B-The average counted a different set of suites either side of this segmentThe average counted a different set of suites either side of this segmentNew registration - model claude-sonnet-5 -> claude-sonnet-5-5, same harness, plan and settingsclaude-sonnet-5 · 2026-W32 · Aug 4, 2026 · 33% · Fclaude-sonnet-5 · 2026-W33 · Aug 11, 2026 · 44% · Dclaude-sonnet-5 · 2026-W34 · Aug 18, 2026 · 41% · D-claude-sonnet-5 · 2026-W35 · Aug 24, 2026 · 43% · Dclaude-sonnet-5 · 2026-W36 · Aug 31, 2026 · 44% · Dclaude-sonnet-5 · 2026-W37 · Sep 7, 2026 · 43% · Dclaude-sonnet-5 · 2026-W38 · Sep 14, 2026 · 46% · Dclaude-sonnet-5 · 2026-W39 · Sep 21, 2026 · 48% · Dclaude-sonnet-5-5 · 2026-W40 · Sep 28, 2026 · 61% · C-The average counted a different set of suites either side of this segmentThe average counted a different set of suites either side of this segmentgpt-5.6-terra · 2026-W32 · Aug 4, 2026 · 28% · Fgpt-5.6-terra · 2026-W33 · Aug 11, 2026 · 48% · Dgpt-5.6-terra · 2026-W34 · Aug 17, 2026 · 57% · C-gpt-5.6-terra · 2026-W35 · Aug 24, 2026 · 44% · Dgpt-5.6-terra · 2026-W36 · Sep 1, 2026 · 44% · Dgpt-5.6-terra · 2026-W37 · Sep 7, 2026 · 55% · D+gpt-5.6-terra · 2026-W38 · Sep 14, 2026 · 51% · D+gpt-5.6-terra · 2026-W39 · Sep 21, 2026 · 51% · D+gpt-5.6-terra · 2026-W40 · Sep 29, 2026 · 49% · D+The average counted a different set of suites either side of this segmentThe average counted a different set of suites either side of this segmentNew registration - model gpt-5.6-sol -> gpt-6-sol and plan ChatGPT Pro 5x -> ChatGPT Plus (account switched back to the operator Plus login 2026-09-23), same harness and settingsNew registration - model gpt-6-sol -> gpt-6.1-sol, same harness, plan and settingsgpt-5.6-sol · 2026-W32 · Aug 4, 2026 · 47% · Dgpt-5.6-sol · 2026-W33 · Aug 11, 2026 · 52% · D+gpt-5.6-sol · 2026-W34 · Aug 17, 2026 · 61% · C-gpt-5.6-sol · 2026-W35 · Aug 24, 2026 · 51% · D+gpt-5.6-sol · 2026-W36 · Aug 31, 2026 · 63% · Cgpt-5.6-sol · 2026-W37 · Sep 7, 2026 · 58% · C-gpt-5.6-sol · 2026-W38 · Sep 14, 2026 · 62% · Cgpt-6-sol · 2026-W39 · Sep 23, 2026 · 51% · D+gpt-6.1-sol · 2026-W40 · Sep 30, 2026 · 67% · Cgpt-6-astra · 2026-W36 · Sep 5, 2026 · 64% · Cgpt-6-astra · 2026-W37 · Sep 7, 2026 · 65% · Cgpt-6-astra · 2026-W38 · Sep 14, 2026 · 64% · Cgpt-6-astra · 2026-W39 · Sep 21, 2026 · 59% · C-gpt-6-astra · 2026-W40 · Sep 28, 2026 · 64% · CThe average counted a different set of suites either side of this segmentThe average counted a different set of suites either side of this segmentNew registration - model gpt-5.6-luna -> gpt-6-luna and plan ChatGPT Pro 5x -> ChatGPT Plus (account switched back to the operator Plus login 2026-09-23), same harness and settingsgpt-5.6-luna · 2026-W32 · Aug 4, 2026 · 47% · Dgpt-5.6-luna · 2026-W33 · Aug 11, 2026 · 51% · D+gpt-5.6-luna · 2026-W34 · Aug 17, 2026 · 49% · D+gpt-5.6-luna · 2026-W35 · Aug 24, 2026 · 53% · D+gpt-5.6-luna · 2026-W36 · Sep 1, 2026 · 50% · D+gpt-5.6-luna · 2026-W37 · Sep 7, 2026 · 47% · Dgpt-5.6-luna · 2026-W38 · Sep 14, 2026 · 53% · D+gpt-6-luna · 2026-W39 · Sep 24, 2026 · 48% · Dgpt-6-luna · 2026-W40 · Sep 27, 2026 · 48% · DThe average counted a different set of suites either side of this segmentThe average counted a different set of suites either side of this segmentcomposer-2.5 · 2026-W32 · Aug 4, 2026 · 57% · C-composer-2.5 · 2026-W33 · Aug 11, 2026 · 51% · D+composer-2.5 · 2026-W34 · Aug 17, 2026 · 48% · Dcomposer-2.5 · 2026-W35 · Aug 24, 2026 · 43% · Dcomposer-2.5 · 2026-W36 · Aug 30, 2026 · 47% · Dcomposer-2.5 · 2026-W37 · Sep 7, 2026 · 47% · Dcomposer-2.5 · 2026-W38 · Sep 13, 2026 · 40% · D-composer-2.5 · 2026-W39 · Sep 20, 2026 · 42% · Dcomposer-2.5 · 2026-W40 · Sep 28, 2026 · 42% · DThe average counted a different set of suites either side of this segmentThe average counted a different set of suites either side of this segmentNew registration - model grok-4.6 -> grok-4.7, same harness, plan and settingsgrok-4.5 · 2026-W32 · Aug 4, 2026 · 60% · C-grok-4.6 · 2026-W33 · Aug 13, 2026 · 66% · Cgrok-4.6 · 2026-W34 · Aug 18, 2026 · 55% · C-grok-4.6 · 2026-W35 · Aug 23, 2026 · 61% · C-grok-4.6 · 2026-W36 · Aug 31, 2026 · 58% · C-grok-4.6 · 2026-W37 · Sep 7, 2026 · 60% · C-grok-4.6 · 2026-W38 · Sep 15, 2026 · 65% · Cgrok-4.7 · 2026-W39 · Sep 26, 2026 · 60% · C-grok-4.7 · 2026-W40 · Sep 28, 2026 · 65% · Ckimi-k3-256k · 2026-W34 · Aug 17, 2026 · 49% · D+kimi-k3-256k · 2026-W35 · Aug 24, 2026 · 43% · Dkimi-k3-256k · 2026-W36 · Aug 31, 2026 · 51% · D+kimi-k3-256k · 2026-W37 · Sep 7, 2026 · 48% · Dkimi-k3-256k · 2026-W38 · Sep 14, 2026 · 53% · D+kimi-k3-256k · 2026-W39 · Sep 22, 2026 · 56% · C-kimi-k3-256k · 2026-W40 · Sep 28, 2026 · 58% · C-

Dotted = the average counted a new suite from that week on. The step is the basis widening, not the agent moving; the ruled week names the suite that joined.

Dotted = a new model version took the line over (grok-4.5 → grok-4.6): the points either side were graded by different models. Plan or settings re-registrations join solid.

gemini-3.1-progemini-3.8-flashclaude-fable-5-1claude-haiku-4-5claude-opus-5-5claude-sonnet-5-5gpt-5.6-terragpt-6.1-solgpt-6-astragpt-6-lunacomposer-2.5grok-4.7kimi-k3-256k

Weekly overall

One row per agent per grading week: the score across every suite it has been graded on, with the plan cost of a full pass. Click a row for the per-suite breakdown.

WeekAgentGradeOverallSuitesTask timeUsage /1w
2026-W40claude-opus-5-5B-now79%3 of 32m 49s2.0% of Claude Max 5x
2026-W40claude-fable-5-1B-now77%3 of 35m 41s8.4% of Claude Max 5x
2026-W40gpt-6.1-solCnow67%3 of 34m 54s6.4% of ChatGPT Plus
2026-W40grok-4.7Cnow65%3 of 319m 47s15.6% of SuperGrok Plus
2026-W40gpt-6-astraCnow64%3 of 32m 40s23.0% of ChatGPT Plus
2026-W40claude-sonnet-5-5C-now61%3 of 31m 39s0.0% of Claude Max 5x
2026-W40kimi-k3-256kC-now58%3 of 35m 33s33.6% of Kimi Allegretto
2026-W40gemini-3.8-flashC-now56%3 of 35m 26s24.8% of Google AI Pro
2026-W40gpt-5.6-terraD+now49%3 of 32m 35s8.6% of ChatGPT Plus
2026-W40gpt-6-lunaDnow48%3 of 31m 16s0.0% of ChatGPT Plus
2026-W40composer-2.5Dnow42%3 of 31m 26s-
2026-W40gemini-3.1-proFnow34%3 of 33m 50s16.6% of Google AI Pro
2026-W40claude-haiku-4-5Fnow26%3 of 33m 7s1.0% of Claude Max 5x
2026-W39claude-opus-5-5C+71%3 of 32m 50s1.2% of Claude Max 5x
2026-W39claude-fable-5-1C-61%3 of 34m 6s11.6% of Claude Max 5x
2026-W39grok-4.7C-60%3 of 318m 31s12.4% of SuperGrok Plus
2026-W39gpt-6-astraC-59%3 of 32m 39s31.0% of ChatGPT Plus
2026-W39kimi-k3-256kC-56%3 of 35m 25s37.0% of Kimi Allegretto
2026-W39gpt-6-solD+51%3 of 32m 44s5.4% of ChatGPT Plus
2026-W39gpt-5.6-terraD+51%3 of 34m 51s10.0% of ChatGPT Plus
2026-W39gpt-6-lunaD48%3 of 31m 15s1.0% of ChatGPT Plus
2026-W39claude-sonnet-5D48%3 of 33m 53s1.0% of Claude Max 5x
2026-W39gemini-3.8-flashD44%3 of 34m 5s21.1% of Google AI Pro
2026-W39composer-2.5D42%3 of 31m 42s-
2026-W39gemini-3.1-proF35%3 of 32m 55s11.7% of Google AI Pro
2026-W39claude-haiku-4-5F31%3 of 33m 4s1.3% of Claude Max 5x
2026-W38claude-fable-5-1C67%3 of 33m 50s14.0% of Claude Max 5x
2026-W38claude-opus-5C66%3 of 35m 5s4.4% of Claude Max 5x
2026-W38grok-4.6C65%3 of 321m 20s14.4% of SuperGrok Plus
2026-W38gpt-6-astraC64%3 of 32m 45s31.0% of ChatGPT Plus
2026-W38gpt-5.6-solC62%3 of 34m 40s17.0% of ChatGPT Plus
2026-W38gpt-5.6-lunaD+53%3 of 32m 31s5.0% of ChatGPT Plus
2026-W38kimi-k3-256kD+53%3 of 32m 35s33.6% of Kimi Allegretto
2026-W38gemini-3.8-flashD+53%3 of 35m 7s21.2% of Google AI Pro
2026-W38gpt-5.6-terraD+51%3 of 32m 37s10.0% of ChatGPT Plus
2026-W38claude-sonnet-5D46%3 of 34m 6s2.0% of Claude Max 5x
2026-W38composer-2.5D-40%3 of 31m 45s-
2026-W38claude-haiku-4-5F30%3 of 33m 24s2.0% of Claude Max 5x
2026-W38gemini-3.1-proF28%3 of 33m 42s13.2% of Google AI Pro
2026-W37claude-fable-5-1C+71%3 of 33m 27s12.4% of Claude Max 5x
2026-W37gpt-6-astraC65%3 of 33m 32s25.0% of ChatGPT Plus
2026-W37claude-opus-5C63%3 of 35m 28s4.0% of Claude Max 5x
2026-W37grok-4.6C-60%3 of 320m 19s2.7% of SuperGrok Plus
2026-W37gemini-3.8-flashC-59%3 of 35m 15s24.6% of Google AI Pro
2026-W37gpt-5.6-solC-58%3 of 33m 32s15.0% of ChatGPT Plus
2026-W37gpt-5.6-terraD+55%3 of 32m 56s11.0% of ChatGPT Plus
2026-W37kimi-k3-256kD48%3 of 36m 5s37.0% of Kimi Allegretto
2026-W37composer-2.5D47%3 of 31m 35s-
2026-W37gpt-5.6-lunaD47%3 of 32m 40s0.0% of ChatGPT Plus
2026-W37claude-sonnet-5D43%3 of 34m 23s2.4% of Claude Max 5x
2026-W37claude-haiku-4-5F31%3 of 33m 47s2.4% of Claude Max 5x
2026-W37gemini-3.1-proF28%3 of 33m 27s14.7% of Google AI Pro
2026-W36claude-fable-5-1B-77%3 of 34m 8s14.4% of Claude Max 5x
2026-W36gpt-6-astraC64%3 of 32m 57s21.0% of ChatGPT Plus
2026-W36gpt-5.6-solC63%3 of 34m 36s16.1% of ChatGPT Plus
2026-W36claude-opus-5C62%3 of 34m 33s4.0% of Claude Max 5x
2026-W36grok-4.6C-58%3 of 320m 31s3.1% of SuperGrok Plus
2026-W36kimi-k3-256kD+51%3 of 35m 31s39.4% of Kimi Allegretto
2026-W36gpt-5.6-lunaD+50%3 of 32m 21s1.1% of ChatGPT Plus
2026-W36gemini-3.8-flashD48%3 of 34m 40s23.3% of Google AI Pro
2026-W36composer-2.5D47%3 of 31m 57s-
2026-W36claude-sonnet-5D44%3 of 33m 48s2.0% of Claude Max 5x
2026-W36gpt-5.6-terraD44%3 of 32m 20s11.7% of ChatGPT Plus
2026-W36claude-haiku-4-5D42%3 of 33m 25s2.0% of Claude Max 5x
2026-W36gemini-3.1-proF35%3 of 33m 14s10.9% of Google AI Pro
2026-W35claude-fable-5C68%3 of 35m 22s14.0% of Claude Max 5x
2026-W35grok-4.6C-61%3 of 317m 10s2.7% of SuperGrok Plus
2026-W35claude-opus-5C-60%3 of 35m 19s4.4% of Claude Max 5x
2026-W35gemini-3.7-flashC-55%3 of 33m 15s10.4% of Google AI Pro
2026-W35gpt-5.6-lunaD+53%3 of 32m 23s1.1% of ChatGPT Plus
2026-W35gpt-5.6-solD+51%3 of 34m 2s14.7% of ChatGPT Plus
2026-W35gpt-5.6-terraD44%3 of 32m 22s10.0% of ChatGPT Plus
2026-W35composer-2.5D43%3 of 32m 28s-
2026-W35kimi-k3-256kD43%3 of 36m 13s42.8% of Kimi Allegretto
2026-W35claude-sonnet-5D43%3 of 34m 35s2.0% of Claude Max 5x
2026-W35gemini-3.1-proD-38%3 of 33m 29s11.1% of Google AI Pro
2026-W35claude-haiku-4-5F26%3 of 33m 26s2.0% of Claude Max 5x
2026-W34claude-fable-5C64%3 of 34m 51s13.2% of Claude Max 5x
2026-W34claude-opus-5C64%3 of 34m 47s4.4% of Claude Max 5x
2026-W34gpt-5.6-solC-61%3 of 33m 28s14.6% of ChatGPT Plus
2026-W34gemini-3.7-flashC-57%3 of 33m 1s13.1% of Google AI Pro
2026-W34gpt-5.6-terraC-57%3 of 32m 17s9.5% of ChatGPT Plus
2026-W34grok-4.6C-55%3 of 315m 37s2.5% of SuperGrok Plus
2026-W34gpt-5.6-lunaD+49%3 of 32m 18s0.9% of ChatGPT Plus
2026-W34kimi-k3-256kD+49%3 of 34m 59s32.0% of Kimi Allegretto
2026-W34composer-2.5D48%3 of 32m 36s-
2026-W34claude-sonnet-5D-41%3 of 34m 45s2.0% of Claude Max 5x
2026-W34claude-haiku-4-5D-37%3 of 33m 46s2.0% of Claude Max 5x
2026-W34gemini-3.1-proF34%3 of 33m 15s10.1% of Google AI Pro
2026-W33gemini-3.7-flashC+72%2 of 33m 1s9.7% of Google AI Pro
2026-W33claude-fable-5C+72%2 of 34m 8s9.2% of Claude Max 5x
2026-W33grok-4.6C66%2 of 37m 9s5.1% of SuperGrok Plus
2026-W33gpt-5.6-solD+52%2 of 33m 15s9.8% of ChatGPT Plus
2026-W33gpt-5.6-lunaD+51%2 of 32m 37s0.9% of ChatGPT Plus
2026-W33composer-2.5D+51%2 of 34m 24s-
2026-W33claude-opus-5D+50%2 of 35m 27s0.0% of Claude Max 5x
2026-W33gpt-5.6-terraD48%2 of 32m 35s7.2% of ChatGPT Plus
2026-W33claude-sonnet-5D44%2 of 35m 14s2.4% of Claude Max 5x
2026-W33gemini-3.1-proF34%2 of 34m 29s8.7% of Google AI Pro
2026-W33claude-haiku-4-5F26%2 of 33m 41s2.0% of Claude Max 5x
2026-W32gemini-3.6-flashC-60%1 of 32m 32s19.6% of Google AI Pro
2026-W32claude-fable-5C-60%1 of 32m 49s8.0% of Claude Max 5x
2026-W32claude-opus-5C-60%1 of 33m 43s3.0% of Claude Max 5x
2026-W32grok-4.5C-60%1 of 34m 7s2.4% of SuperGrok Plus
2026-W32composer-2.5C-57%1 of 31m 57s-
2026-W32gpt-5.6-lunaD47%1 of 31m 35s0.7% of ChatGPT Plus
2026-W32gemini-3.1-proD47%1 of 32m 31s16.8% of Google AI Pro
2026-W32gpt-5.6-solD47%1 of 33m 31s11.4% of ChatGPT Plus
2026-W32claude-sonnet-5F33%1 of 33m 48s1.0% of Claude Max 5x
2026-W32claude-haiku-4-5F30%1 of 33m 9s1.0% of Claude Max 5x
2026-W32gpt-5.6-terraF28%1 of 31m 26s4.4% of ChatGPT Plus