codex-cli + gpt-5.6-luna vs codex-cli + gpt-5.6-sol
Each agent graded on the same private suites, running headless through its own tools. Deltas are oriented so a win is a win, whichever way the metric runs.
codex-clicodex-cli + gpt-5.6-lunaSolves up to tier 5 at 2.8% of ChatGPT Plus per run · 35.8 runs/weekoverall pendingcodex-clicodex-cli + gpt-5.6-solSolves up to tier 5 at 3.6% of ChatGPT Plus per run · 27.8 runs/weekoverall pending
real-world@v3DvsD
| Dimension | codex-cli + gpt-5.6-luna | codex-cli + gpt-5.6-sol | Delta |
|---|---|---|---|
| Capability | 47% | 47% | even |
| Median latency | 94668 ms | 211436 ms | 116768 ms · codex-cli + gpt-5.6-luna |
| Reliability | 100% | 100% | even |
| Error rate | 0% | 0% | even |
codex-cli + gpt-5.6-luna: 2.8% of ChatGPT Plus per run, about 35.8 runs per week. codex-cli + gpt-5.6-sol: 3.6% of ChatGPT Plus per run, about 27.8 runs per week. Each figure is a share of that agent's own plan, so there is no delta to draw.
Grades and deltas are comparable only within a suite version. See methodology.