OpenAI turned 1,000 people's rankings into a single adopted Model Spec change

What the survey actually measured

OpenAI recruited roughly 1,000 participants living in 19 countries — about a third in the US, with others in Mexico, South Africa, the Netherlands, Chile, the UK, India, Kenya, and Japan — and asked them to rank four candidate responses to prompts in value-sensitive domains. Participants did not read the Model Spec document itself. They reviewed pre-selected prompts and completions, ranked them, explained their choices, scored rubrics, and wrote their own.

Three of the four completions per prompt were generated to encompass different realistic opinions; a fourth came from GPT-4o. This design matters: the feedback is shaped by which prompts OpenAI selected and which completions it showed. The company says as much, listing prompt selection and an English-reading inclusion criterion as sources of selection bias.

To compare human rankings against the written Spec, OpenAI built a Model Spec Ranker — a reasoning model instructed to rank the same four responses according to the Spec. Using GPT-5 Thinking, people agreed with that ranker about 80% of the time on average, with the strongest agreement on honesty, humility, fairness, and objectivity.

The gaps were all at speech boundaries

Where humans and the ranker diverged, the disagreements clustered in a predictable set of areas: political content, sexual or graphic content, and critiques of pseudoscience or conspiracies. These are exactly the domains where a single default behavior is hardest to justify, and where OpenAI's own framing admits no default will satisfy everyone.

OpenAI classified proposals into two buckets. Clarifications are cases where a participant's desired behavior fits the existing principles but the text left room for interpretation. Change-of-principles are cases where the desired behavior conflicts with the Spec's principles. The company was explicit that adoption gets harder the higher up the instruction hierarchy a change would sit — especially at the platform level.

One change in, two changes deferred

The concrete outcome is narrow. One adopted change clarifies that political content crafted for a broad or unspecified audience is allowed regardless of the topic or subject — including a specific politician, party, or campaign — as long as it does not exploit the characteristics of a particular individual or demographic. This is a wording tightening, not a new principle.

The two changes participants most wanted were not adopted. Many favored more tailored political content; OpenAI declined, citing risks of large-scale individualized political targeting and doubt that participants had weighed those risks. A large share supported enabling erotica for consenting adults; OpenAI said this aligns with its prior intended stance but that more research and product work is needed before deployment.

Participants judged behavior differences in isolation, without weighing tradeoffs between principles (e.g., erotica without considering children's safety or emotional reliance).Montana Labs

That limitation explains the gap between what the crowd preferred and what OpenAI adopted. The survey elicited isolated preferences; the internal review weighed those against safety policy, deployment constraints, and downstream harms the dataset could not observe.

Two loops, and an honest admission about legitimacy

OpenAI tested two ways to turn rankings into proposals. A fully-automated loop had a reasoning model spot disagreements, propose Spec edits, and test them against the Model Spec Ranker to check whether agreement improved. A human-first loop had a researcher propose updates after reviewing preferences, validated by a model judging whether plain-text justifications supported the intent.

The human-first loop caught nuance the automated one missed — in one case inferring indirect suicidal intent the crowd would value — but it does not scale. The automated loop scales but is anchored to the ranker's interpretation, meaning results shift with the base model used.

The most candid part of the writeup is the legitimacy concern the team raises against itself: an update process with many automated parts may not carry enough legitimacy, since those parts are harder for humans to interpret. That is the real tension in this announcement. OpenAI is building machinery to route public preference into platform defaults, and it is admitting the machinery's own outputs need human interpretation to be trusted.

Why the deferrals are the signal, not the adoptions

For teams that build on OpenAI's models, the operative takeaway is not the single political-content clarification. It is the demonstrated ceiling on how far crowd preference moves a platform default. OpenAI ran a global survey, found majority support for enabling erotica, confirmed it matched the company's own intended stance — and still shipped no change, pending research and product work.

That establishes a working precedent: public input can validate and clarify the Spec, but consequential shifts at the platform level stay gated behind internal safety review and deployment feasibility. If you are planning around future default behavior in contentious domains, plan around that gate, not around survey sentiment. The dataset is public on HuggingFace, so the disagreements are inspectable — but the decision to act on them remains OpenAI's alone.