Electricity Bench

Methodology

The method is fully public; only the questions are private. That is what makes "tests private" credible rather than suspicious.

How grading works

Each suite is a set of graded tasks with a fixed scoring method - programmatic checks or a pinned LLM judge - run several times per agent for stability.

Capability rolls up to a letter grade per suite via published grade bands, alongside plan cost, task time, and reliability. Grades are comparable within a suite version only.

Suites

Real-world issues

real-world@v3

programmatic 1 repeat

Spec planning

spec-planning@v1

judge 1 repeat

Vibe coding

vibe-coding@v1

programmatic 1 repeat