Jul, 134 min to read
Frontend
News · OpenAI's LifeSciBench grades free-response science answers against 19,020 expert rubric criteria
Jul, 84 min to read
FrontendOpenAI's LifeSciBench grades free-response science answers against 19,020 expert rubric criteria
The benchmark drops clean multiple-choice formats for open-ended collaborator requests, then scores them line by line — and the results expose where frontier models still stall.
The request replaces the question
Most life science evaluations, as OpenAI describes them, reduce research to fact-recall or clean prediction problems with structured formats and tidy reference answers. LifeSciBench discards that shape. Each of its 750 tasks is built to look like something a scientist would actually say to a capable colleague: a scientific prompt, any relevant context or attached artifacts, and a free-response answer.
The published example makes the format concrete. A team preparing for a Type B FDA meeting asks for a 'hard-nosed critique' of whether their AAV9 micro-dystrophin package supports accelerated approval on a surrogate endpoint. The prompt supplies biopsy Western blot numbers, immunofluorescence readings, a 48-week NSAA change, safety events, and eligibility criteria, then asks the model to pressure-test the package item by item. There is no checkbox answer — the output is expected to read like a reviewer's memo.
This is a deliberate interface decision. By framing the input as a working request rather than a quiz item, LifeSciBench measures whether a model can operate inside the messy front-end of research — incomplete evidence, conflicting results, decisions under uncertainty — instead of the sanitized version benchmarks usually test.
Grading that mirrors how scientists judge each other
The other half of this design is the rubric. Across the benchmark, expert-developed rubrics carry 19,020 criteria — an average of 25 per task. The DMD example is scored against six weighted criteria totaling 100 points, with 24 points for catching the assay-quantification problems (MANEX1A epitope sharing, invalid full-length dystrophin standards), 22 for explaining why expression level is not automatically a valid clinical surrogate, and only 8 for noting patient-selection and sample-size gaps.
OpenAI's stated reason is that a correct high-level conclusion can still be judged incomplete — if a response overlooks a key assay limitation or fails to raise a consequential biological nuance. Conversely, a partial answer can contain high-quality reasoning without solving the task. The benchmark reports two metrics to capture this: pass rate, the share of tasks clearing a 70% task-level threshold, and score, the average rubric reward that grants partial credit.
The validation numbers suggest the format holds up. Feedback came from 453 reviewers not involved in writing tasks; 97% held a doctorate, with an average of 12 years of experience and 14 peer-reviewed publications. Agreement exceeded 96% in every category, with 90.4% strongly agreeing that tasks reflect real-world work.
The artifact gap
The most telling result sits at the seam between the prompt and its attachments. LifeSciBench includes 1,062 attached artifacts — figures, PDFs, tables, sequence files, structure and chemical files, web references — and 53% of tasks require interpreting or synthesizing at least one of them.
Performance falls off sharply when models must read those artifacts rather than the prompt text alone. GPT-Rosalind drops from 45.1% on text-only tasks to 28.1% when artifacts or URLs are involved; GPT-5.5 shows the same shape, 29.9% down to 21.9%. OpenAI attributes the gap to models struggling to extract information from complex figures or large sequence files and fold it into the final answer.
Exact-output formats are worse still. GPT-Rosalind reaches only 14.8% on numeric tasks and 24.0% on sequence or structure outputs, with construct-generation tasks at 27.3% and barely improved over GPT-5.5. These are the outputs that would be used directly — CRISPR/HDR donor design, siRNA design — where a small formatting or calculation slip drops a response below threshold.
Partial credit is not a usable deliverable
GPT-Rosalind lifts overall exact pass rate from 25.7% to 36.1%, with its strongest gains in scientific communication (56.3% to 71.1%, though n=9) and translation (36.8% to 57.7%). Those are the categories with a clear evidence boundary that reward organizing and explaining. Design, optimization, and prediction (30.7%) and analysis (30.3%) remain the hardest.
The specific implication of this benchmark's rubric-plus-pass-rate design is what it exposes about the space between the two scores. In roughly 14% of tasks, models earned substantial rubric credit while failing the exact-pass threshold; for GPT-Rosalind, 109 tasks scored under 20% pass rate yet earned at least 50% rubric reward. A model can surface the right evidence and write a plausible partial answer, then miss a single constraint, use the wrong data, or fail to connect its reasoning to a usable decision.
For anyone building on these systems, that is the number to internalize: a response that looks 50% correct on a rubric is still, in operational terms, an unfinished draft a scientist must audit. LifeSciBench measures the front-end interaction — request in, expert-facing answer out — and its verdict is that frontier models are becoming persuasive collaborators well before they become dependable ones.
Find this story relevant to you?
Contact us to find a unique solution
Need an AI engineering partner that can actually build?
We help businesses integrate AI, build AI-powered products, automate high-value workflows, and modernize the software systems behind them.
Related reading
More analysis around product delivery, operational AI, and the systems work that makes deployment hold up in reality.