Jul, 134 min to read
Platform
News · OpenAI's GDPval scores frontier models against 14-year professionals across 44 occupations
Jul, 84 min to read
PlatformOpenAI's GDPval scores frontier models against 14-year professionals across 44 occupations
A benchmark built from GDP data down to individual job tasks — and one where a competitor's model came out on top.
Building a benchmark from GDP downward
Most benchmarks start from what a model can be asked to do. GDPval starts from where economic value sits and works backward to the model. OpenAI began with the nine U.S. industries each contributing over 5% of GDP (using Federal Reserve Bank of St. Louis data), took the five highest-wage occupations in each from the May 2024 BLS occupational employment report, then filtered to occupations that are at least 60% knowledge work per O*NET task classifications.
That funnel produced 44 occupations spanning software developers, lawyers, registered nurses, mechanical engineers, pharmacists, financial analysts and film editors. The full set holds 1,320 tasks — 30 per occupation — with 220 (five per occupation) released as an open gold set.
The tasks themselves are not text prompts. Each comes with reference files and expects real deliverables: documents, slides, diagrams, spreadsheets, multimedia. The published manufacturing-engineer example asks for a jig concept design produced in 3D modeling software and delivered as a PDF built from PowerPoint snapshots. Each task was written by professionals averaging 14 years of experience and passed through roughly five rounds of expert review.
Blind grading, and a competitor at the top
Grading relies on expert reviewers from the same occupations comparing model output against the task writer's own solution — blind, not knowing which deliverable is human. Each model output is classified as better than, as good as, or worse than the human work, guided by occupation-specific rubrics the task writers built.
OpenAI ran this against GPT-4o, o4-mini, o3, GPT-5, Claude Opus 4.1, Gemini 2.5 Pro and Grok 4. The notable result: Claude Opus 4.1, an Anthropic model, was the best performer, rated as good as or better than experts on just under half the tasks, excelling on aesthetics like formatting and layout. GPT-5 led on accuracy, particularly domain-specific knowledge.
Publishing a benchmark where a rival wins is unusual, and it reframes GDPval as a platform play rather than a marketing chart. OpenAI is releasing the gold task subset and a public grading service — including an experimental automated grader trained to predict expert judgments, which it explicitly says is not yet reliable enough to replace human graders.
Reading the 100x claim
OpenAI reports that frontier models complete GDPval tasks roughly 100x faster and 100x cheaper than industry experts. The company attaches an unusually direct caveat: those figures reflect only model inference time and API billing rates, not the human oversight, iteration, and integration required to actually use the output in a workplace.
The second caveat is structural. GDPval is one-shot. It does not capture revising a legal brief after client feedback, iterating on an analysis after spotting an anomaly, or the ambiguity of deciding a brief is even the right deliverable. A lawyer, OpenAI notes, might have to talk to a client before knowing what to produce.
So the 100x is a floor on cost and a ceiling on completeness at once: cheap generation of a first artifact, measured against a professional's finished single output, with the entire back-and-forth of real work excluded.
What a task-level benchmark leaves for the buyer
GDPval's most useful contribution to applied teams is its granularity. It scores tasks, not jobs, and OpenAI is explicit that most jobs are more than collections of writable-down tasks. That distinction is the buying decision: the benchmark tells you where a model produces expert-comparable one-shot deliverables, not where it can own a workflow.
The controlled experiments point the same way — larger models, more reasoning steps, and richer task context each produced measurable gains, and an internally retrained GPT-5 improved further. Context and iteration move the numbers, which is precisely the surface OpenAI admits GDPval doesn't yet measure.
The practical implication of this specific release: treat GDPval scores as a map of which discrete deliverables to route to a model first, while budgeting separately for the oversight, drafting rounds, and ambiguity resolution that the one-shot format deliberately omits. The next versions promising interactivity and ambiguous tasks will be the ones worth benchmarking a real deployment against.
Find this story relevant to you?
Contact us to find a unique solution
Need an AI engineering partner that can actually build?
We help businesses integrate AI, build AI-powered products, automate high-value workflows, and modernize the software systems behind them.
Related reading
More analysis around product delivery, operational AI, and the systems work that makes deployment hold up in reality.