Seeing at the limit: diffraction, sampling, and what a camera can never see

I’ve spent twenty years building systems that extract meaning from images, and the most useful fact I know about them comes not from machine learning but from nineteenth-century optics: there is a hard physical limit on what any camera can resolve, and no algorithm can honestly recover what diffraction has destroyed.

Understanding where that limit comes from changes how you design vision systems — what you promise, what hardware you specify, and when you should say “no model can do that.”

Light doesn’t travel in straight lines

Geometric optics — the ray diagrams from school — says a perfect lens focuses a point of light to a point. Wave optics says otherwise. Light passing through any finite aperture spreads, because a wave squeezed through an opening interferes with itself. A perfect lens with a circular aperture focuses a distant point source not to a point but to a bullseye pattern — the Airy disk — with angular radius

$$\theta \approx 1.22\,\frac{\lambda}{D},$$

where $\lambda$ is the wavelength of light and $D$ is the aperture diameter. Two stars, two pixels of a license plate, two cells under a microscope — if their angular separation is less than $\theta$, their Airy patterns overlap and they merge into one blob. That is the Rayleigh criterion, and it’s not an engineering limitation. It’s interference, the same physics as the double-slit experiment.

In microscopy the same wall appears as the Abbe diffraction limit: the smallest resolvable feature is

$$d = \frac{\lambda}{2\,\mathrm{NA}},$$

with $\mathrm{NA}$ the numerical aperture of the objective. Green light at $\lambda \approx 550\,\text{nm}$ through a high-quality oil-immersion objective ($\mathrm{NA} \approx 1.4$ — beyond the reach of any dry lens, since $\mathrm{NA} = n \sin\theta$ caps below 1 in air) gives $d \approx 200\,\text{nm}$. That number is why optical microscopes stalled for a century, and why beating it (with tricks like STED and PALM, which sidestep the assumptions rather than the physics) earned the 2014 Nobel Prize in Chemistry.

The imaging equation

There’s a cleaner way to say all of this. An imaging system doesn’t transmit the scene; it transmits the scene convolved with its own response. If $O$ is the true scene and $h$ is the system’s point-spread function (the Airy pattern above), the image is

$$I(x, y) = (O * h)(x, y) + n(x, y),$$

where $n$ is noise. Take the Fourier transform and convolution becomes multiplication:

$$\tilde{I}(f) = \tilde{O}(f)\,H(f) + \tilde{n}(f).$$

$H(f)$, the optical transfer function, is the aperture acting as a low-pass filter: it rolls off with spatial frequency and hits exactly zero beyond a cutoff $f_c = D/(\lambda z)$. Beyond that frequency the scene’s detail isn’t attenuated — it is multiplied by zero. Gone. Deconvolution can rebalance frequencies where $H(f)$ is small but nonzero; where $H(f) = 0$, there is nothing to rebalance. Any “detail” an algorithm paints there is prior, not measurement — a plausible hallucination. For an artistic upscaler, fine. For a medical image or a license-plate readout presented as evidence, it is fabrication.

The sensor gets a vote too

Optics isn’t the only limit. The sensor samples the image on a pixel grid, and the Nyquist–Shannon theorem says a signal sampled at spacing $p$ can only represent spatial frequencies below

$$f_{\text{Nyquist}} = \frac{1}{2p}.$$

Detail finer than that doesn’t vanish politely — it aliases, folding down into false coarse patterns (the moiré you see when someone wears a fine-striped shirt on camera). A well-designed camera matches the pixel pitch to the optical cutoff: pixels much larger than the Airy disk waste the lens; pixels much smaller waste silicon on blur. When you choose a camera for a vision system, you are choosing which physics limit you’d rather hit first.

Photons are countable, and that’s a problem

One more wall, the quietest one: light arrives as photons, and photon arrivals are a Poisson process. Collect $N$ photons in a pixel and the fluctuation is $\sqrt{N}$, so the best possible signal-to-noise ratio is

$$\mathrm{SNR} = \frac{N}{\sqrt{N}} = \sqrt{N}.$$

This is shot noise — not sensor imperfection, but the granularity of light itself. It’s why low-light footage is grainy no matter what you paid for the camera, and why every denoiser is ultimately making educated guesses about photons it never received.

Why this matters for AI

Modern computer vision is spectacular at exploiting every photon and every spatial frequency the physics delivers. What it cannot do — what nothing can do — is recover information that diffraction filtered out, sampling aliased, or shot noise drowned. The imaging equation draws a sharp line between restoration (inverting what the system attenuated) and generation (inventing what it never captured).

The practical discipline is to know, for any vision problem, which side of the line you’re on: compute the diffraction spot, the Nyquist limit, and the photon budget before promising what the model will see. The best vision systems I’ve worked on were designed by people who respected those three numbers. The worst failures came from teams that assumed software could out-argue physics.

It can’t. It can only get closer to the limit — which, done well, is more than enough to build remarkable things.