News
Agent scorecards, suite releases, and grading-run announcements. Also available as an RSS feed.
GPT-6.1 Sol is good, just two weeks too lateGraded a day after release: GPT-6.1 Sol fixes more than GPT-6 Sol on a little less usage, but Opus 5.5 and Sonnet 5.5 now beat the Sol and Luna pair.Claude Sonnet 5.5 is a Sonnet done rightGraded on release day: Claude Sonnet 5.5 is the fastest agent on the bench, light on the plan and strong on well-defined work.Claude pulls further ahead as GPT-6 agents climb - Week 40Claude Opus 5.5 leads Week 40 ahead of Fable 5.1 as the GPT-6 agents improve, Google's agents slow down and Grok stays far slower than the rest.Grok 4.7 looks like Grok 4.6 with the effort dial movedGraded as the successor to Grok 4.6: at high effort Grok 4.7 ran too long to finish our suites, and once lowered to medium it lands on the same grades as 4.6 at the same slow pace.GPT-6 Sol is a step back from 5.6, Luna stands stillGraded two days after release: GPT-6 Sol fixes less than GPT-5.6 Sol on well under half the usage, and GPT-6 Luna is faster but not better.Claude Opus 5.5 takes the top at a fraction of Fable's usageGraded a day after release: Claude Opus 5.5 tops the bench, matches Fable 5.1 on plans and uses a fraction of Fable's plan usage.Claude Opus 5 takes the top as most of the field slips - Week 39Claude Opus 5 leads Week 39, seven of 13 agents slip a step on the same tasks, Fable 5.1 still writes the best plans, and Kimi K3 is slow again.Claude holds the top as GPT-6 Astra closes in - Week 38 resultsWeek 38 splits the top: Claude Opus 5 is the pick for real repository work, GPT-6 Astra for plans, Claude Fable 5.1 stays the safest all-round subscription.Claude Fable 5.1 beats out GPT-6 Astra - Week 37 agent resultsThe first weekly grading run after the release of Claude Fable 5.1, GPT-6 Astra and Gemini 3.8 Flash side by side.GPT-6 Astra: no better than Sol, worse than FableGraded on release day: GPT-6 Astra matches GPT-5.6 Sol on our suites, runs faster, burns more of its plan, and stays behind Claude Fable 5.1.Gemini 3.8 Flash: no step up, more quota, more waitingGemini 3.8 Flash graded the day it shipped. It solves no more than 3.7 Flash, takes about twice as long and burns about two and a half times the Google AI Pro quota. Staying on 3.7 Flash makes the most sense for now.Claude Fable 5.1 takes the top spot on the benchGraded hours after release: Claude Fable 5.1 is the first agent to earn an overall B, fixing more than Claude Fable 5 and spending less of the plan to do it.Claude gets faster and keeps the lead - Week 36 AI agent resultsClaude Fable 5 stays in front and speeds up, and OpenAI's new 5-hour cap on the $20 Plus plan changes the plan math.Claude Fable 5 keeps the lead - Week 35 resultsA quiet week: Claude Fable 5 stays the agent to beat, Gemini 3.7 Flash takes the real-world top spot, Grok 4.6 gets slower, and Kimi K3 burns more quota.Claude Fable 5 leads Week 34 as spec-planning debutsA new suite tests whether agents can plan before they code, Kimi K3 joins the board, and Claude Fable 5 leads by having no weak spot.Gemini 3.7 Flash: more capability for less quotaGemini 3.7 Flash solves more real-world tasks than 3.6 Flash and uses less Google AI Pro quota - but it takes longer.New suite: spec-planning - did the spec see the bug coming?There is a new suite on the bench: spec-planning. It measures something none of our other suites do: can a coding agent look at a real codebase, read a big...Grok 4.6, released today, ties Fable 5 for the vibe-coding leadxAI's Grok 4.6 shipped and graded the same day: it ties Claude Fable 5 for the vibe-coding lead, doubling Grok 4.5's task time and quota burn along the way.Claude Fable 5 leads vibe-coding - Week 33 AI agent resultsEleven coding agents across five harnesses faced real-world and vibe-coding suites; Claude Fable 5 shared the real-world lead and won vibe-coding.New suite: vibe-coding - how much of a bug fix is the bug report?We cut a new private suite today: vibe-coding. It grades something none of our other suites isolate: how well a coding agent fixes a real bug as the bug...Week 2026-W32 on the bench: the first grading run, 10 coding agents on real subscriptionsThe leaderboard now shows what it was built to show: real agent harnesses, on the subscriptions people actually pay for, graded on the same private task...Why honest numbers are hardWhy evaluating AI coding agents fairly is challenging, and how Electricity Bench handles quota metering, independent grades, and version baselines.How we gradeHow a grade rolls up on Electricity Bench: evaluating the complete coding agent on real subscriptions across capability, task duration, and plan quota.Introducing Electricity BenchElectricity Bench grades AI coding agents on private task suites and real subscription plans, publishing independent scorecards and weekly leaderboards.