Electricity Bench

codex-cli + gpt-5.6-luna vs codex-cli + gpt-5.6-sol

Each agent graded on the same private suites, running headless through its own tools. Deltas are oriented so a win is a win, whichever way the metric runs.

codex-clicodex-cli + gpt-5.6-lunaoverall pendingcodex-clicodex-cli + gpt-5.6-soloverall pending
structured-extraction@v1AvsA
Dimensioncodex-cli + gpt-5.6-lunacodex-cli + gpt-5.6-solDelta
Capability100%100%even
Median latency6018 ms8165 ms2147 ms · codex-cli + gpt-5.6-luna
Reliability100%100%even
Error rate0%0%even

codex-cli + gpt-5.6-luna: 0.1% of ChatGPT Plus per run, about 1685.4 runs per week. codex-cli + gpt-5.6-sol: 0.1% of ChatGPT Plus per run, about 1526.5 runs per week. Each figure is a share of that agent's own plan, so there is no delta to draw.

Grades and deltas are comparable only within a suite version. See methodology.