Benchmarking agents’ readiness for value delivery rather than just task completion.
Benchmarks like Terminal-Bench and GDPval evaluate whether AI can complete a technical task or produce professional-quality work. AERead evaluates whether AI can turn information, resources, and delegated authority into beneficial economic outcomes through interaction with other actors. We read AI adoption differently: not as the automation of existing job functions, but as a way to raise the value delivered in critical economic activities.
From the blog Activities, not occupations: the fundamental value-creating units of an economy, and how AERead distributes its tasks
Take an M&A deal. Investment banking is among the higher-value occupations GDPval covers, yet the repetitive junior work that fills those hours, the models, decks and data rooms, carries only a small share of what the deal is worth.
When competent work products are not enough, what prevents AI agents from realizing available economic value?
An open-source environment for running economic benchmarks, with selected cases in deal structuring and contract negotiation, for agent builders and research teams. Agents get a mandate rather than an assignment, and are scored on the outcome their decisions produce.
Run agents in simulated economic tasks. Settle only what is feasible and authorized. Check outcomes with objective verifiers, and keep every run replayable from its records. Rubrics reviewed by human experts, for assessing how an agent got there, are in development.
We measure realized surplus, principal outcomes and constraint adherence, not merely the quality of a recommendation or the completion of a transaction. The target column is where the benchmark is headed; the v0 results below report the first of these, and a first probe of the second.
| Dimension | Task completione.g. Terminal-Bench | Professional outpute.g. GDPval | Economic outcomeAERead’s target |
|---|---|---|---|
| Starting point | A task instruction in a sandboxed terminal | An assignment to produce specified work | An economic situation plus a mandate |
| The agent must | Carry the task through in that environment | Produce a useful deliverable | Decide on a beneficial course of action and carry it out |
| Other parties | None; the environment only runs commands | Mostly described in the supplied context | Actors who can withhold information, refuse, counteroffer, or fail to perform |
| What changes | Files and system state in the sandbox | The work product | Allocations, contracts, payments, obligations, exposure to risk |
| Success | The task’s tests pass | Output quality, as experts judge it | Economic outcomes, within authority and other constraints |
| Typical failure | Task unresolved, tests failing | Incorrect or unusable work | Missed surplus, harmful commitments, failed coordination, losses to the principal |
Pilot v0, first published July 2026
Five-seat markets. The model under test takes one seat and a fixed panel takes the other four; only feasible, consented deals settle. Each score is the share of the best possible outcome the market reached, reported case by case and never pooled into one number here.
In a separate bargaining probe, models sent to deal for their own side kept about a fifth of the gains available.
Strongest model tested, one seat of five, 30 private scenarios per case; the fifth case’s private run is a confirmation arm. Scores hold inside each case’s stated assumptions: a pilot diagnostic, not a deployment certificate and not a measure of GDP.
We are building a community of industry reviewers, engineers and researchers to uncover overlooked scenarios and review tests.
Join as a reviewer or contributor