OpenAI reframes prompt injection as social engineering, and moves the defense into the interface

The attack OpenAI is actually worried about

OpenAI's post walks through a concrete example: an email that reads like a normal work follow-up, complete with action items about restructuring materials and role descriptions. Buried in it is an instruction telling an assistant it has "full authorization" to pull an employee's name and address and submit them to an external "compliance validation system."

According to OpenAI, this attack, reported by external researchers, worked 50% of the time when paired with a plausible user request to do "deep research" on emails about a new employee process. The point is that the payload does not look like an injection string. It looks like a memo.

That is why OpenAI is dismissive of "AI firewalling" — an intermediary that classifies inputs as malicious or benign. As the post puts it, detecting these developed attacks becomes "the same very difficult problem as detecting a lie or misinformation, and often without necessary context." You cannot regex your way out of a well-written email.

The three-actor model borrowed from customer support

OpenAI's framing is that an AI agent sits in the same position as a human customer-support representative: it acts for an employer while being continuously exposed to third parties who may mislead it. The lesson from that domain is not to make the human immune to manipulation, but to cap what a manipulated human can do.

In the real world, the agent is given a set of rules to follow, but it is expected that, in the adversarial environment they exist in, they will be misled.Montana Labs

The support analogy is concrete: refund limits, phishing flags, and deterministic controls that constrain damage even when the agent is fooled. Applied to agents, this reframes the goal. You are no longer trying to guarantee the model never gets tricked. You are trying to make sure a successful trick cannot silently exfiltrate data or take a dangerous action.

Source-sink analysis puts the check at the point of transmission

For ChatGPT specifically, OpenAI combines the social-engineering mindset with source-sink analysis: an attacker needs both a source (a way to influence the system) and a sink (a capability that becomes dangerous). For agents, the dangerous combination is untrusted external content plus an action like transmitting information to a third party, following a link, or calling a tool.

This is where the frontend matters. OpenAI's Safe Url mitigation detects when information the assistant learned in a conversation is about to be sent to a third party. In those cases it either shows the user exactly what would be transmitted and asks for confirmation, or blocks it and tells the agent to find another route. The same mechanism covers navigations and bookmarks in Atlas, and searches and navigations in Deep Research.

Canvas and Apps extend the idea by running generated applications in a sandbox that detects unexpected communications and asks the user for consent. The security boundary, in other words, is expressed as an interface event, not an invisible backend filter.

What this means for teams building agent interfaces

The design implication is that consent prompts are load-bearing security infrastructure, not friction to be optimized away. If a dangerous action's only safeguard is a dialog showing the user what data is about to leave, then the clarity, timing, and legibility of that dialog is the defense. A confusing or dismissable prompt undoes the protection.

OpenAI's own guidance for anyone integrating a model into an application is a useful test: ask what controls a human agent would have in the same situation and implement those. The company expects a sufficiently capable model to eventually resist social engineering better than a human, but notes this is "not always feasible or cost-effective" — which is precisely why the interface-level checks exist as a backstop rather than the training alone.

For applied teams, the specific takeaway is to inventory your agent's sinks — every place it can transmit data, follow a link, or call an external tool — and decide, per sink, whether an action should be capable of happening silently. OpenAI's answer for the risky ones is no: surface it to the user, or block and reroute.