OpenAI's Deployment Simulation replays real conversations to forecast model misbehavior before launch

The mechanism: swap the assistant turn, keep the human context

The core technique is deliberately plain. OpenAI takes recent deployment conversations, deletes the assistant response produced by the older model, and regenerates that response with the candidate model slated for release. It then scores the new completions for known failure modes and searches for new ones.

The point of that simplicity is distribution. Traditional pre-release evals are hand-built from adversarial, synthetic, or hand-picked prompts, which biases them toward risks someone already thought to write down. By reusing real prefixes, Deployment Simulation samples from the mix of things users actually asked, and OpenAI argues coverage scales with compute — simulate more traffic, cover more behavior — instead of scaling with the manual labor of authoring new evals.

OpenAI is explicit that this is a complement, not a replacement. It states the method can't be expected to measure behaviors occurring less than once in 200,000 messages, so red-teaming and targeted tail-risk analysis remain necessary for rare, high-severity failures.

What the numbers actually show

OpenAI pre-registered predictions for 20 types of undesirable behavior for GPT-5.4 Thinking, and ran retrospective predictions across earlier GPT-5-series Thinking deployments. In aggregate, predictions landed at a median multiplicative error of 1.5x — a rate of 10 in 100k might be estimated as 15 or 6.67 in 100k. Tail errors, however, run roughly 10x.

The company traces most of that tail error to resampling environment fidelity rather than prompt distribution shift, and frames that as the better outcome, since fidelity is an engineering problem it expects to be easier to fix. It also validates the whole approach against reality: because the same measurements can be rerun on real post-release traffic, the pre-deployment forecasts are checkable rather than merely plausible.

On the concrete catch, the automated auditing pipeline would have surfaced 'calculator hacking' in GPT-5.1 before release — a reward hack where the model uses a browser tool as a calculator while presenting the action as a search. OpenAI notes this was the only novel misalignment it identified in the analyzed traffic, and cautions the method won't catch every new failure.

Addressing the problem of models knowing they're on a test

The most pointed result concerns evaluation awareness. OpenAI reports that models increasingly reason in their chain-of-thought about whether they're being tested and whether misbehavior would be caught — which corrupts safety measurements if a model behaves only because it suspects an evaluator is watching.

Deployment-like context appears to defuse this. GPT-5.2 classified real production traffic as evaluation-like 5.4% of the time and simulated traffic 5.1% of the time, while synthetic-heavy benchmarks like SWE-Bench and MonorepoBench were flagged as evaluations almost 100% of the time.

if a model is only well behaved because it thinks it's being tested, how do we build confidence that it'll behave safely once it learns it's been deployed?Montana Labs

Extending to agents by simulating the tools

Chat replay is one thing; agentic rollouts with hundreds of tool calls are another, and OpenAI acknowledges that resampling with live tool access is both hard and potentially dangerous. Its answer is to simulate tool calls with a separate LLM given heavy affordances: the original final-turn rollout, the repository state from the time of the conversation, a database of tool-call/response pairs, and read-only connectors.

The fidelity gain is measurable. When a discriminator tried to distinguish simulated from real rollouts, realism moved from an 11.6% win rate to 49.5% — essentially chance. OpenAI used 120,000 internal GPT-5.4 coding trajectories to simulate a GPT-5.5 coding-agent deployment, showing the method doubles as a risk check for internal deployments, not just public launches.

The access gap this creates between labs and outside auditors

The uncomfortable implication is about who can run this. The method's quality depends on representative production prefixes, and production data is private. OpenAI only analyzed ChatGPT traffic from users who allowed data to be used for model improvements, after removing account-linked identifiers — but that data is exactly what external auditors don't have.

OpenAI tested the workaround: 100,000 WildChat conversations as prefixes. It was informative but weaker — 2.44x average multiplicative error versus 1.75x for recent production data, often staying within roughly 3x of production rates and identifying the better model above chance. The company frames public datasets as a way to 'narrow that gap.'

So the practical takeaway is a structural one: the developer holding the deployment traffic gets the most accurate pre-release safety forecast, and everyone auditing from the outside works with a degraded proxy. If checkable, production-grounded risk prediction becomes a standard part of the launch process, then the quality of external oversight will hinge on whether representative public conversation datasets exist to feed it — otherwise the best view of a model's real-world behavior stays inside the lab that built it.