Grades over time
Overall leads: each agent's score across every suite it has been graded on, one point per grading week. Switch to a single suite for its own line. Filter by provider or agent, click a point for the breakdown.
Overall is the capability mean across an agent's graded suites - the same rule the leaderboard's overall grade uses - with each suite contributing its latest reading, so a week that re-ran one suite moves the line by that suite's share of the mean rather than replacing it. The week a new suite joined the average is ruled off on the chart. Grades compare only within a suite version. Plan usage is each run's share of that agent's own subscription over one week, as the provider reports it - every agent is read on the same window, whatever window its provider declares, because the weekly limit is what these plans are sold on. The usage tab plots those shares side by side, but a percent of one plan is not a percent of another, so read it per agent, not as a ranking. The chart does not mark provider announcements; the dates of every plan and limit change, with the usage measured either side of each, live on the plan change log.
Real-world issues · Spec planning · Vibe coding · 13 agents · every suite counted, each contributing its latest grade
Dotted = the average counted a new suite from that week on. The step is the basis widening, not the agent moving; the ruled week names the suite that joined.
Dotted = a new model version took the line over (grok-4.5 → grok-4.6): the points either side were graded by different models. Plan or settings re-registrations join solid.
Weekly overall
One row per agent per grading week: the score across every suite it has been graded on, with the plan cost of a full pass. Click a row for the per-suite breakdown.
| Week | Agent | Grade | Overall | Usage /1w |
|---|---|---|---|---|
| 2026-W40 | claude-opus-5-5 | B-now | 79% | 2.0% of Claude Max 5x |
| 2026-W40 | claude-fable-5-1 | B-now | 77% | 8.4% of Claude Max 5x |
| 2026-W40 | gpt-6.1-sol | Cnow | 67% | 6.4% of ChatGPT Plus |
| 2026-W40 | grok-4.7 | Cnow | 65% | 15.6% of SuperGrok Plus |
| 2026-W40 | gpt-6-astra | Cnow | 64% | 23.0% of ChatGPT Plus |
| 2026-W40 | claude-sonnet-5-5 | C-now | 61% | 0.0% of Claude Max 5x |
| 2026-W40 | kimi-k3-256k | C-now | 58% | 33.6% of Kimi Allegretto |
| 2026-W40 | gemini-3.8-flash | C-now | 56% | 24.8% of Google AI Pro |
| 2026-W40 | gpt-5.6-terra | D+now | 49% | 8.6% of ChatGPT Plus |
| 2026-W40 | gpt-6-luna | Dnow | 48% | 0.0% of ChatGPT Plus |
| 2026-W40 | composer-2.5 | Dnow | 42% | - |
| 2026-W40 | gemini-3.1-pro | Fnow | 34% | 16.6% of Google AI Pro |
| 2026-W40 | claude-haiku-4-5 | Fnow | 26% | 1.0% of Claude Max 5x |
| 2026-W39 | claude-opus-5-5 | C+ | 71% | 1.2% of Claude Max 5x |
| 2026-W39 | claude-fable-5-1 | C- | 61% | 11.6% of Claude Max 5x |
| 2026-W39 | grok-4.7 | C- | 60% | 12.4% of SuperGrok Plus |
| 2026-W39 | gpt-6-astra | C- | 59% | 31.0% of ChatGPT Plus |
| 2026-W39 | kimi-k3-256k | C- | 56% | 37.0% of Kimi Allegretto |
| 2026-W39 | gpt-6-sol | D+ | 51% | 5.4% of ChatGPT Plus |
| 2026-W39 | gpt-5.6-terra | D+ | 51% | 10.0% of ChatGPT Plus |
| 2026-W39 | gpt-6-luna | D | 48% | 1.0% of ChatGPT Plus |
| 2026-W39 | claude-sonnet-5 | D | 48% | 1.0% of Claude Max 5x |
| 2026-W39 | gemini-3.8-flash | D | 44% | 21.1% of Google AI Pro |
| 2026-W39 | composer-2.5 | D | 42% | - |
| 2026-W39 | gemini-3.1-pro | F | 35% | 11.7% of Google AI Pro |
| 2026-W39 | claude-haiku-4-5 | F | 31% | 1.3% of Claude Max 5x |
| 2026-W38 | claude-fable-5-1 | C | 67% | 14.0% of Claude Max 5x |
| 2026-W38 | claude-opus-5 | C | 66% | 4.4% of Claude Max 5x |
| 2026-W38 | grok-4.6 | C | 65% | 14.4% of SuperGrok Plus |
| 2026-W38 | gpt-6-astra | C | 64% | 31.0% of ChatGPT Plus |
| 2026-W38 | gpt-5.6-sol | C | 62% | 17.0% of ChatGPT Plus |
| 2026-W38 | gpt-5.6-luna | D+ | 53% | 5.0% of ChatGPT Plus |
| 2026-W38 | kimi-k3-256k | D+ | 53% | 33.6% of Kimi Allegretto |
| 2026-W38 | gemini-3.8-flash | D+ | 53% | 21.2% of Google AI Pro |
| 2026-W38 | gpt-5.6-terra | D+ | 51% | 10.0% of ChatGPT Plus |
| 2026-W38 | claude-sonnet-5 | D | 46% | 2.0% of Claude Max 5x |
| 2026-W38 | composer-2.5 | D- | 40% | - |
| 2026-W38 | claude-haiku-4-5 | F | 30% | 2.0% of Claude Max 5x |
| 2026-W38 | gemini-3.1-pro | F | 28% | 13.2% of Google AI Pro |
| 2026-W37 | claude-fable-5-1 | C+ | 71% | 12.4% of Claude Max 5x |
| 2026-W37 | gpt-6-astra | C | 65% | 25.0% of ChatGPT Plus |
| 2026-W37 | claude-opus-5 | C | 63% | 4.0% of Claude Max 5x |
| 2026-W37 | grok-4.6 | C- | 60% | 2.7% of SuperGrok Plus |
| 2026-W37 | gemini-3.8-flash | C- | 59% | 24.6% of Google AI Pro |
| 2026-W37 | gpt-5.6-sol | C- | 58% | 15.0% of ChatGPT Plus |
| 2026-W37 | gpt-5.6-terra | D+ | 55% | 11.0% of ChatGPT Plus |
| 2026-W37 | kimi-k3-256k | D | 48% | 37.0% of Kimi Allegretto |
| 2026-W37 | composer-2.5 | D | 47% | - |
| 2026-W37 | gpt-5.6-luna | D | 47% | 0.0% of ChatGPT Plus |
| 2026-W37 | claude-sonnet-5 | D | 43% | 2.4% of Claude Max 5x |
| 2026-W37 | claude-haiku-4-5 | F | 31% | 2.4% of Claude Max 5x |
| 2026-W37 | gemini-3.1-pro | F | 28% | 14.7% of Google AI Pro |
| 2026-W36 | claude-fable-5-1 | B- | 77% | 14.4% of Claude Max 5x |
| 2026-W36 | gpt-6-astra | C | 64% | 21.0% of ChatGPT Plus |
| 2026-W36 | gpt-5.6-sol | C | 63% | 16.1% of ChatGPT Plus |
| 2026-W36 | claude-opus-5 | C | 62% | 4.0% of Claude Max 5x |
| 2026-W36 | grok-4.6 | C- | 58% | 3.1% of SuperGrok Plus |
| 2026-W36 | kimi-k3-256k | D+ | 51% | 39.4% of Kimi Allegretto |
| 2026-W36 | gpt-5.6-luna | D+ | 50% | 1.1% of ChatGPT Plus |
| 2026-W36 | gemini-3.8-flash | D | 48% | 23.3% of Google AI Pro |
| 2026-W36 | composer-2.5 | D | 47% | - |
| 2026-W36 | claude-sonnet-5 | D | 44% | 2.0% of Claude Max 5x |
| 2026-W36 | gpt-5.6-terra | D | 44% | 11.7% of ChatGPT Plus |
| 2026-W36 | claude-haiku-4-5 | D | 42% | 2.0% of Claude Max 5x |
| 2026-W36 | gemini-3.1-pro | F | 35% | 10.9% of Google AI Pro |
| 2026-W35 | claude-fable-5 | C | 68% | 14.0% of Claude Max 5x |
| 2026-W35 | grok-4.6 | C- | 61% | 2.7% of SuperGrok Plus |
| 2026-W35 | claude-opus-5 | C- | 60% | 4.4% of Claude Max 5x |
| 2026-W35 | gemini-3.7-flash | C- | 55% | 10.4% of Google AI Pro |
| 2026-W35 | gpt-5.6-luna | D+ | 53% | 1.1% of ChatGPT Plus |
| 2026-W35 | gpt-5.6-sol | D+ | 51% | 14.7% of ChatGPT Plus |
| 2026-W35 | gpt-5.6-terra | D | 44% | 10.0% of ChatGPT Plus |
| 2026-W35 | composer-2.5 | D | 43% | - |
| 2026-W35 | kimi-k3-256k | D | 43% | 42.8% of Kimi Allegretto |
| 2026-W35 | claude-sonnet-5 | D | 43% | 2.0% of Claude Max 5x |
| 2026-W35 | gemini-3.1-pro | D- | 38% | 11.1% of Google AI Pro |
| 2026-W35 | claude-haiku-4-5 | F | 26% | 2.0% of Claude Max 5x |
| 2026-W34 | claude-fable-5 | C | 64% | 13.2% of Claude Max 5x |
| 2026-W34 | claude-opus-5 | C | 64% | 4.4% of Claude Max 5x |
| 2026-W34 | gpt-5.6-sol | C- | 61% | 14.6% of ChatGPT Plus |
| 2026-W34 | gemini-3.7-flash | C- | 57% | 13.1% of Google AI Pro |
| 2026-W34 | gpt-5.6-terra | C- | 57% | 9.5% of ChatGPT Plus |
| 2026-W34 | grok-4.6 | C- | 55% | 2.5% of SuperGrok Plus |
| 2026-W34 | gpt-5.6-luna | D+ | 49% | 0.9% of ChatGPT Plus |
| 2026-W34 | kimi-k3-256k | D+ | 49% | 32.0% of Kimi Allegretto |
| 2026-W34 | composer-2.5 | D | 48% | - |
| 2026-W34 | claude-sonnet-5 | D- | 41% | 2.0% of Claude Max 5x |
| 2026-W34 | claude-haiku-4-5 | D- | 37% | 2.0% of Claude Max 5x |
| 2026-W34 | gemini-3.1-pro | F | 34% | 10.1% of Google AI Pro |
| 2026-W33 | gemini-3.7-flash | C+ | 72% | 9.7% of Google AI Pro |
| 2026-W33 | claude-fable-5 | C+ | 72% | 9.2% of Claude Max 5x |
| 2026-W33 | grok-4.6 | C | 66% | 5.1% of SuperGrok Plus |
| 2026-W33 | gpt-5.6-sol | D+ | 52% | 9.8% of ChatGPT Plus |
| 2026-W33 | gpt-5.6-luna | D+ | 51% | 0.9% of ChatGPT Plus |
| 2026-W33 | composer-2.5 | D+ | 51% | - |
| 2026-W33 | claude-opus-5 | D+ | 50% | 0.0% of Claude Max 5x |
| 2026-W33 | gpt-5.6-terra | D | 48% | 7.2% of ChatGPT Plus |
| 2026-W33 | claude-sonnet-5 | D | 44% | 2.4% of Claude Max 5x |
| 2026-W33 | gemini-3.1-pro | F | 34% | 8.7% of Google AI Pro |
| 2026-W33 | claude-haiku-4-5 | F | 26% | 2.0% of Claude Max 5x |
| 2026-W32 | gemini-3.6-flash | C- | 60% | 19.6% of Google AI Pro |
| 2026-W32 | claude-fable-5 | C- | 60% | 8.0% of Claude Max 5x |
| 2026-W32 | claude-opus-5 | C- | 60% | 3.0% of Claude Max 5x |
| 2026-W32 | grok-4.5 | C- | 60% | 2.4% of SuperGrok Plus |
| 2026-W32 | composer-2.5 | C- | 57% | - |
| 2026-W32 | gpt-5.6-luna | D | 47% | 0.7% of ChatGPT Plus |
| 2026-W32 | gemini-3.1-pro | D | 47% | 16.8% of Google AI Pro |
| 2026-W32 | gpt-5.6-sol | D | 47% | 11.4% of ChatGPT Plus |
| 2026-W32 | claude-sonnet-5 | F | 33% | 1.0% of Claude Max 5x |
| 2026-W32 | claude-haiku-4-5 | F | 30% | 1.0% of Claude Max 5x |
| 2026-W32 | gpt-5.6-terra | F | 28% | 4.4% of ChatGPT Plus |