OpenAI rebuilt its WebRTC stack around a stateless relay that routes on the ICE ufrag

The constraint that forced the redesign: WebRTC's port model versus Kubernetes

OpenAI's team frames the problem in operational terms rather than product ones. Serving more than 900 million weekly active users meant the conventional WebRTC deployment pattern — one public UDP port per session — stopped fitting their infrastructure.

They spell out why. At high concurrency, one-port-per-session forces you to expose and manage very large public UDP port ranges. Cloud load balancers and Kubernetes services aren't built around tens of thousands of public UDP ports. That surface is hard to secure, hard to audit, and hostile to autoscaling, since pods are constantly added, removed, and rescheduled while each would need to reserve and advertise a stable port range.

The obvious fix — a single UDP port per server with application-level demultiplexing — solves port count but creates a second problem. ICE and DTLS are stateful, so the process that created a session must keep receiving that session's packets. Across a load-balanced fleet, the first packet can still land on the wrong instance.

Routing on a field that already exists in the handshake

OpenAI's answer splits packet routing from protocol termination. A lightweight Go relay with a small public footprint sits in front of a stateful transceiver that owns the full WebRTC session — ICE connectivity checks, the DTLS handshake, SRTP keys, and lifecycle. The relay never decrypts media or runs ICE state machines; it reads just enough packet metadata to pick a destination.

The clever move is where the routing hint lives. Rather than adding a hot-path lookup service, OpenAI encodes routing metadata into the ICE username fragment (ufrag), a short identifier already exchanged during setup and echoed in STUN binding requests. The transceiver generates a server-side ufrag carrying enough information to identify the destination cluster and owning transceiver.

So the client's first media-path packet — usually a STUN binding request sent to a shared relay VIP like 203.0.113.10:3478 — is parsed by the relay, which decodes the ufrag and forwards it to the right transceiver. Subsequent DTLS, RTP, and RTCP packets flow without re-decoding. If a relay restarts, the next STUN packet rebuilds the mapping from the ufrag; a Redis cache holds the client-IP-to-transceiver mapping so recovery can happen even earlier.

What the browser never notices

For a frontend engineer, the defining property of this work is negative: the client sees nothing new. OpenAI is explicit that from the client's perspective, nothing about the WebRTC session changes. The transceiver still presents a normal WebRTC flow and still owns all protocol state; only the internal routing changed.

For us, that meant changing the shape of our WebRTC deployment without changing what clients expect from WebRTC itself.Montana Labs

This is a deliberate design principle they list as "preserve protocol semantics at the edge." Because clients keep speaking standard WebRTC, browser and mobile interoperability stays intact — the team gets to reshape its own infrastructure without shipping custom client behavior or fragmenting support across platforms. The complexity was pushed into a thin routing layer, not into every backend service and not into the client.

The post also credits the WebRTC ecosystem it builds on, noting that Justin Uberti, one of WebRTC's original architects, and Sean DuBois, creator of the Pion Go implementation, are now OpenAI colleagues. The first transceiver implementation was a single Go service built on Pion.

Choosing the boring option: no SFU, no kernel bypass

Two decisions stand out as restraint. First, OpenAI rejected the SFU (selective forwarding unit) that many teams default to. An SFU makes sense for multiparty calls, but OpenAI's traffic is mostly 1:1 — one user to one model — and latency-sensitive on every turn. Their relay-plus-transceiver result, they write, confirmed an SFU-less design was the right default and let inference services scale like ordinary services instead of acting as WebRTC peers.

Second, they declined kernel bypass. The relay runs entirely in userspace as a regular Go process, tuned with SO_REUSEPORT to spread packets across workers, runtime.LockOSThread to pin flows to CPU cores for cache locality, and pre-allocated buffers to avoid garbage-collection overhead. That was enough to handle global real-time media traffic with a relatively small relay footprint, so they kept the simpler design.

Geography is layered on top: Global Relay places ingress points near users to shorten the first hop, and Cloudflare geo and proximity steering routes signaling to a nearby transceiver cluster, with the ufrag ensuring media still lands on the anchored transceiver.

The takeaway: routing metadata belongs in the protocol you already run

The specific implication of this announcement is that the ICE ufrag can double as a deterministic first-packet router. By embedding routing hints into a field WebRTC already carries, OpenAI got session-sticky forwarding across a fleet without a hot-path lookup dependency and without breaking standard client behavior.

For teams building real-time voice on WebRTC, the reusable lesson isn't the transceiver or the relay by name — it's the sequencing. Solve the deployment mismatch (UDP ports versus Kubernetes) with a thin stateless forwarding layer, keep the hard stateful pieces in exactly one place, and reach for exotic tooling like kernel bypass only after the common case demands it. OpenAI's path suggests it usually doesn't.