Jul, 134 min to read
Frontend
News · OpenAI's FrontierScience benchmark separates olympiad answers from open-ended research work
Jul, 84 min to read
FrontendOpenAI's FrontierScience benchmark separates olympiad answers from open-ended research work
A new expert-written science benchmark scores GPT-5.2 at 77% on olympiad-style problems but only 25% on multi-step research tasks graded by rubric.
Two tracks, two very different scores
FrontierScience is not one benchmark but two, and the split is the whole point. The Olympiad track holds 100 short-answer questions written by international olympiad medalists, gradable by a number, expression, or fuzzy string match. The Research track holds 60 multi-step subtasks written by PhD scientists and graded against a 10-point rubric, where a solution counts as correct only if it earns at least 7 points.
On the Olympiad track, GPT-5.2 scored 77%, with Gemini 3 Pro close behind at 76%. On the Research track, GPT-5.2 scored 25%. Same model, same domains of physics, chemistry, and biology — a 52-point gap that tracks the difference between arriving at a verifiable final answer and reasoning correctly through an open-ended problem.
OpenAI reads this the way scientists already use the tools: models can support structured reasoning now, while humans still frame the problem and validate the output. The 25% figure is the more honest number for what remains hard.
The rubric is the real engineering artifact
The interesting build here is the grading architecture for the Research track. Rather than scoring only a final answer, each question ships with a rubric of multiple independent, objectively assessable items totaling 10 points, and those items credit intermediate reasoning steps — not just the conclusion. The sample chemistry rubric awards separate points for correctly analyzing traditional synthesis limitations, describing the thiolate-mediated tetramerization, interpreting NMR ring-current data, and so on.
This lets the benchmark distinguish a model that guessed the right endpoint from one that reasoned its way there, and it enables failure analysis at the step level. OpenAI notes the tradeoff plainly: multi-component rubrics on long tasks are less objective than checking a single final answer.
The catch is who does the grading. Expert human grading does not scale, so responses are scored by a model-based grader — GPT-5 — checking against the rubric criteria. The rubric was deliberately designed to be checkable by that grader. So a GPT-family model is evaluating frontier models, including its own successor, against expert-written criteria. OpenAI built a verification pipeline to calibrate difficulty and correctness, but the dependence on a model grader is a structural feature worth naming.
How the benchmark defends against contamination and self-favoring
Two design choices push against the obvious objections. First, contamination: the full set spans over 700 questions, but only 160 are in the open-sourced gold set (100 Olympiad, 60 Research). The rest are held out specifically to track contamination over time.
Second, self-favoring during construction. Task creation included selection against OpenAI's internal models — questions the models already solved were discarded. OpenAI states outright that this should bias the evaluation against its own models relative to competitors. That admission matters when reading GPT-5.2's top placement, since Gemini 3 Pro effectively tied on the Olympiad set despite that bias.
The scale of expert involvement is concrete: 42 former medalists or national coaches contributing 109 olympiad medals for the Olympiad track, and 45 doctoral candidates, postdocs, and professors for the Research track, spanning fields from quantum electrodynamics to synthetic organic chemistry to evolutionary biology. Tasks moved through four stages — Creation, Review, Resolution, Revision — with independent expert cross-review.
What the 25% Research score tells builders
For teams deploying models against real scientific work, the useful signal is not the headline 77% but the shape of the failures. OpenAI reports that on transcripts, frontier models made reasoning, logic, and calculation errors, misunderstood niche concepts, and produced factual inaccuracies. Longer thinking time raised accuracy for both GPT-5.2 and o3, which suggests some failures are compute-bound rather than capability-bound.
The stated limits are the boundary of the claim: FrontierScience uses constrained problem statements, does not test hypothesis generation, does not touch video, other modalities, or real experimental systems. It measures reasoning on expert-written problems, not how science actually gets done. GPQA's trajectory — GPT-4 at 39% in November 2023, GPT-5.2 at 92% two years later — is the reason OpenAI expects this benchmark to saturate too, and why the Research track's rubric grading exists to buy headroom.
The practical implication of FrontierScience is a specific one: it gives a step-resolved way to see where a model's chemistry or physics reasoning breaks mid-derivation, rather than only whether it landed the final answer. That is the resolution applied teams need if they want to trust a model on a research subtask — and the 25% score is a direct warning against trusting it unattended on the open-ended ones.
Find this story relevant to you?
Contact us to find a unique solution
Need an AI engineering partner that can actually build?
We help businesses integrate AI, build AI-powered products, automate high-value workflows, and modernize the software systems behind them.
Related reading
More analysis around product delivery, operational AI, and the systems work that makes deployment hold up in reality.