Meta Adds Text Prompts and Single-Image 3D to the Segment Anything Line

From clicking a pixel to typing 'red baseball cap'

The core change in SAM 3 is how you tell the model what to segment. SAM 1 and SAM 2 relied on visual prompts — you pointed at something in the frame. SAM 3 accepts detailed text prompts and, per Meta, segments every matching object across an image or video.

Meta frames this against a specific weakness in prior systems: models with fixed label sets could handle a coarse concept like 'bus' or 'car' but stumbled on 'yellow school bus.' SAM 3 is pitched as handling that longer tail of descriptions, and Meta says it can be paired with multimodal LLMs for compound prompts like 'people sitting down, but not wearing a red baseball cap.'

That negation example matters more than the color adjective. Excluding a subset within a scene requires reasoning about relationships between objects, not just matching a phrase to a mask — which is why Meta routes those cases through an external MLLM rather than SAM 3 alone.

SAM 3D's single-image bet, and the dataset built to grade it

SAM 3D is two open source models: SAM 3D Objects for object and scene reconstruction, and SAM 3D Body for human body and shape estimation. Both work from a single image, which is the harder and more useful constraint — no multi-view capture rig required.

Meta claims SAM 3D Objects significantly outperforms existing methods, but the more telling detail is that they built the yardstick alongside the model. SAM 3D Artist Objects is an evaluation dataset assembled with artists, described as a first-of-its-kind, more rigorous way to measure 3D reconstruction progress.

Shipping a new benchmark with a new model is a double-edged move: it acknowledges that prior 3D benchmarks were too easy to be meaningful, while also letting the releasing lab define the terms of its own comparison. The value depends on whether outside teams adopt the benchmark.

The Playground and the in-house product pipeline

Meta is exposing both models through Segment Anything Playground, a no-code web tool where users upload an image or video and prompt with a text phrase, or start from templates. The practical templates are worth noting — pixelating faces, license plates, and screens — alongside the playful ones like motion trails and spotlight effects.

The named product integrations show where Meta intends the models to earn their keep: object- and person-specific effects coming to the Edits video app, creation experiences in Vibes on the Meta AI app and meta.ai, and a View in Room feature on Facebook Marketplace that places 3D reconstructions of items like lamps or tables into a buyer's space.

What Meta is actually releasing to developers

The release is uneven across the two models, and the details matter for anyone planning to build on them. For SAM 3, Meta is sharing model weights, an open-vocabulary segmentation benchmark dataset, and a research paper describing how the model was built. For SAM 3D, it is sharing model checkpoints and inference code plus a new 3D reconstruction benchmark.

Notably, the announcement mentions weights for SAM 3 but checkpoints and inference code for SAM 3D — with no explicit mention of SAM 3D training code or a paper. Teams should verify the exact licensing and artifacts before assuming full reproducibility on the 3D side.

Meta is also partnering with Roboflow so users can annotate data and fine-tune SAM 3 for their own tasks. That partnership signals Meta expects SAM 3 to be adapted rather than used off the shelf — the base model is a starting point for domain-specific segmentation.

The real signal: open-vocabulary segmentation as a labeled component

For applied teams, the concrete takeaway is that SAM 3 collapses two steps — detecting a named object and producing a pixel mask for it — into one text-prompted call, with a fine-tuning path through Roboflow when the built-in vocabulary falls short.

That changes the build-versus-integrate calculus for anything involving video editing, redaction, or asset extraction. The pixelate-faces-and-plates templates hint at compliance and privacy pipelines; the Marketplace View in Room feature shows 3D reconstruction moving from research demo to a shipping consumer flow. The open question is whether the accompanying benchmarks are adopted widely enough to make Meta's performance claims verifiable outside Meta.