CloudSignal AI

Blog

Depth from anything: monocular 3D and the road to spatial AI

By Ashwin Rajendraprasad Computer Vision

Depth used to require two cameras or a laser. Monocular depth estimation has gotten good enough that a single RGB image yields usable 3D geometry. Depth Anything 3, from ByteDance’s Seed team (project page), shows how far a deliberately simple model can push it. This is the substrate for “spatial AI”: robots, AR, and 3D commerce all need to know where surfaces are.

Depth is the missing sense

Most computer-vision stacks are flat. A detector tells you what and where in the image, but not how far. A startling amount of product behavior depends on that missing axis. Placing a virtual object convincingly, measuring a room, letting a robot avoid a table, understanding that a person is in front of the shelf rather than painted on it: all of it needs depth. For years, getting depth meant dedicated hardware (stereo rigs, structured light, LiDAR), with all the cost, calibration, and bulk that implies. The appeal of monocular depth is obvious: every phone and every existing camera becomes a potential depth sensor.

Monocular depth, and why it used to be unreliable

Recovering depth from a single image is, strictly, ill-posed: infinitely many 3D scenes can produce the same 2D picture. Humans do it anyway, using learned cues: perspective, occlusion, texture gradients, familiar sizes. Learned models do the same. The catch, historically, was reliability. Early monocular models produced depth maps that looked plausible but wobbled: inconsistent across frames, unstable across scenes, fine on the examples they were tuned for and untrustworthy in the wild. Good enough for a photo effect, not for anything that had to make a decision.

The arc: from V2 to a deliberately plain model

Depth Anything V2 (arXiv:2406.09414, June 2024) was a notable step toward robust, general monocular depth. Depth Anything 3 (DA3), released by ByteDance Seed in 2025 with an arXiv report and an ICLR 2026 oral paper, is the one worth studying, partly for its results and partly for a design choice that runs against instinct.

The instinct in this field is to add machinery: task-specific heads, geometry modules, architectural special-casing for cameras and poses. DA3 goes the other way. Its core is a single plain transformer trained toward a single depth-ray target, with, in the team’s framing, “no architectural specialization.” The backbone is a vanilla DINOv2 encoder, the same Meta self-supervised line we covered in the DINOv3 piece. The bet is that a strong, general representation plus a clean training objective beats a pile of bespoke components. It’s the lesson that keeps recurring in modern deep learning: scale and simplicity, applied to a good representation, tend to win.

What DA3 can do, and how to read its numbers

The capabilities are what make this a spatial-AI story rather than a depth-map story. DA3 does single-view-to-multi-view reconstruction, produces multi-view-consistent geometry without known camera poses, and supports pose-free SLAM, direct prediction of 3D Gaussian Splatting parameters, and the multi-camera perception that autonomous vehicles need. “Consistent geometry from any number of views, no poses required” is the phrase to hold onto: it’s what lets depth feed the rest of a 3D pipeline instead of sitting in a corner as a pretty heat map.

Now the numbers, carefully — because this is a good lesson in reading model claims. The DA3 team reports that the model surpasses the prior state of the art, VGGT (the Visual Geometry Grounded Transformer, from Oxford’s Visual Geometry Group and Meta AI), on camera-pose and geometric accuracy, while matching Depth Anything V2 on monocular detail. But the authors’ own two sources give different improvement figures. The DA3 project page states roughly 35.7% (camera pose) and 23.6% (geometry); the arXiv paper (2511.10647) states roughly 44.3% and 25.1%. Same result, two sets of numbers, depending on which document you open. These are self-reported improvements over VGGT, so attribute them to the team rather than treating them as settled fact, and if you cite a figure, say which source it came from. That the primary sources disagree with each other is exactly why you benchmark on your own data before betting a product on a headline percentage.

Practitioner notes: relative vs. metric, and where it fails

Two things decide whether monocular depth is usable for your task.

The first is relative versus metric depth. Relative depth tells you that A is closer than B; metric depth tells you A is 1.9 meters away. A huge number of applications (measurement, robotics, try-on sizing) need real-world units, and getting reliable metric scale from a single image is the harder problem. Know which one your pipeline actually requires before you assume a model delivers it.

The second is failure cases. Monocular depth leans on visual cues, so it struggles exactly where those cues lie: reflective and transparent surfaces (glass, mirrors, polished metal) confidently return wrong geometry, and textureless or ambiguous regions get guessed. And “a single plain transformer” is not a small model; there’s a real compute and latency cost, which matters the moment you move from an offline batch to a live camera on a device.

Licensing decides whether you can ship at all, and with DA3 it depends on which checkpoint you pull. The code is Apache-2.0, but the released weights split: DA3-Giant, DA3-Large, and the DA3Nested-Giant-Large models carry CC BY-NC 4.0 and are therefore non-commercial, while DA3-Base, DA3-Small, DA3Metric-Large, and DA3Mono-Large are Apache-2.0. A commercial build starts from that second group, so run your benchmarks on those checkpoints rather than assuming the headline numbers carry down from the giant ones.

Depth as a building block

The reason to care about all this isn’t depth for its own sake; it’s what depth enables. Reliable geometry from ordinary cameras is the foundation layer for virtual try-on, in-scene measurement, and spatial understanding. It’s also what feeds newer 3D pipelines like feed-forward Gaussian splatting, which turns predicted geometry into renderable scenes. Depth is the input that makes those downstream systems possible.

The practitioner’s posture: treat DA3 as a strong, general starting point, verify metric scale and failure behavior on your own scenes, and design for the compute it needs. If you’re prototyping spatial AI (try-on, measurement, 3D capture), depth estimation is the first building block to get right, and it’s a productive place to start an architecture conversation.

Sources / further reading

Written by Ashwin Rajendraprasad for CloudSignal AI.