Computer Vision · entry 05/05

Vision in the wild

Every vision system is trained on one distribution of images and deployed on another — the gap has a taxonomy, a monitoring discipline, and a body count of confident demos.

The distribution you trained on isn't coming

Benchmark images are curated: centered subjects, decent light, web-photographer framing. Deployment is dusk, rain on the lens, motion blur, a camera mounted two meters higher than any training shot, and a sensor with different color response. Accuracy drops the moment a model leaves its training distribution, usually quietly — the model still outputs confident labels, just wronger ones. This isn't a bug to fix once; it's the permanent condition of fielded vision, the deployment version of the generalization problem.

Shortcut learning

Models learn whatever separates the training classes most cheaply — which is often not the thing you meant. The classic cases are real: cows recognized by green pasture (a cow on a beach becomes a dog), huskies distinguished from wolves by snow in the background, a medical model reading the ruler that only appeared in malignant photos. Training rewarded the correlation; nothing required the concept. Spot-check what your model attends to on out-of-context examples before trusting it anywhere important — the shortcut works until the day it doesn't, and that day is unannounced.

The world also moves

Even a model honestly matched to deployment decays. Seasons change the foliage, the store rearranges shelves, the product gets a redesign, the camera drifts out of focus, a new phone model changes image processing. Fielded vision is an operations practice: log a sample of production frames, review a slice with humans weekly, track confidence and class-frequency distributions for drift, and budget periodic retraining as a running cost, not a one-time capital fix. The eval-first habit carries the whole practice: your monitoring set is just an eval that never stops running.

Adversaries, where they exist

Beyond accidents, vision can be attacked: printed patches that hide a person from a detector, stickers that turn a stop sign into a speed limit for one specific model, crafted noise invisible to humans. Robustness research helps at the margins, but the honest posture is architectural: don't let a single camera frame authorize anything irreversible — the same containment logic as agent design. Where no adversary exists, weather will do the attacking for free.

Failure mode

Validating once, deploying forever. The launch eval was clean; two years later the model is confidently wrong about a world that redesigned itself, and nobody noticed because nothing crashed — accuracy failures in vision are silent by default. If there is no dashboard tracking live performance on sampled, human-checked frames, the real accuracy is unknown, and "unknown" in production means "lower than you think."