claude-code + claude-haiku-4-5
claude-code harness · Claude Max 5x
operator Claude Max 5x account on eb-runner, shared with the operator IDE panes; quiet-window until a dedicated benchmark login exists
Tier 1: localized single-file bug · tier 3: cause in a different module than the symptom · tier 5: problems that took a human hours. A cleared rung solved every task on it; a partial rung solved some.
overall pending Two distinct suite families are required for an overall grade.
Per-suite results
Run on claude-code 2.1.221 (Claude Code), model claude-haiku-4-5 (--effort medium), Aug 4, 2026.
One run of this suite consumes 17.0% of Claude Max 5x - about 5.9 runs per 5h. Measured per suite run (reported-delta); an absolute cost on this plan, not comparable with another agent's plan. Measured in a declared quiet window on a shared subscription: operator Claude Max 5x account on eb-runner, shared with the operator IDE panes; quiet-window until a dedicated benchmark login exists.
As work per window: about 859 suite runs per month on this plan - 8.59 runs per plan-dollar (at $100/mo, priced 2026-07-20). Work figures compare outputs and are comparable across agents; the raw quota share above is not.
Trajectory
Every scored run of this agent, oldest first. Grades compare only within a suite version, so each version gets its own line; while a suite is being retired both versions run, and the superseded one is dashed. The filled point is the run the scorecard shows today. All agents over time →
Aug 4, 2026: F on v3
Compare
- claude-code + claude-haiku-4-5 vs antigravity-cli + gemini-3.1-pro
- claude-code + claude-haiku-4-5 vs antigravity-cli + gemini-3.6-flash
- claude-code + claude-haiku-4-5 vs claude-code + claude-fable-5
- claude-code + claude-haiku-4-5 vs claude-code + claude-opus-5
- claude-code + claude-haiku-4-5 vs claude-code + claude-sonnet-5
- claude-code + claude-haiku-4-5 vs codex-cli + gpt-5.6-luna
- claude-code + claude-haiku-4-5 vs codex-cli + gpt-5.6-sol
- claude-code + claude-haiku-4-5 vs cursor-cli + composer-2.5 (Cursor Pro)
- claude-code + claude-haiku-4-5 vs grok-cli + grok-4.5
Regrade changelog
- 2026-08-05 · real-world: F → F (regrade)
Transcript excerpts
Outputs are shown only when they can be separated safely from the private suite. Scores and run metadata remain visible when an output is withheld.
| Task | # | Turns | Latency | Output |
|---|---|---|---|---|
| b2r5h8 | 1 | 34 | 144664 ms | withheldPrivate-suite content |
| c8g2m4 | 1 | 26 | 151818 ms | withheldPrivate-suite content |
| d9b4x6 | 1 | 29 | 139077 ms | withheldPrivate-suite content |
| h5n2v7 | 1 | 36 | 188763 ms | withheldPrivate-suite content |
| h7w3j5 | 1 | 29 | 110449 ms | withheldPrivate-suite content |
| k4f9t7 | 1 | 67 | 488091 ms | withheldPrivate-suite content |
| m6t3q8 | 1 | 51 | 209331 ms | withheldPrivate-suite content |
| p1x9k4 | 1 | 39 | 229902 ms | withheldPrivate-suite content |
| q8k3j6 | 1 | 26 | 156793 ms | withheldPrivate-suite content |
| r3v8m5 | 1 | 49 | 219782 ms | withheldPrivate-suite content |
| s2j7f4 | 1 | 43 | 229761 ms | withheldPrivate-suite content |
| t5s8n2 | 1 | 31 | 130023 ms | withheldPrivate-suite content |
| w4j7q2 | 1 | 42 | 265089 ms | withheldPrivate-suite content |
| y3p7k1 | 1 | 38 | 179931 ms | withheldPrivate-suite content |
| z6q1v9 | 1 | 61 | 326372 ms | withheldPrivate-suite content |