The Computation Was Pushed to the Data

Pushed by physics; fixed in the infrastructure.

The plan is reasonable, and almost everyone draws it the same way. Put a camera on a drone, fly the route, send the video to a server, run the analysis there: count the things you care about, flag the events you are watching for. Someone helpfully points out that you do not even need results in real time, the report can land a few minutes later. With the hardest requirement gone, the streaming part feels like the safe, easy part of the project. In the demo, on the office network, it works.

Then you fly it over the actual site, and the analysis quietly comes apart: not with an error, but with wrong answers. The counts drift. The events go missing. Nothing in the logs raises its hand. It takes a while to accept that the model was not the main problem. The video was. Over a moving wireless link, frames were being lost in transit, and you cannot analyze frames you never received. And the damage runs deeper than missing data: analysis that tracks motion depends on the continuity between frames, and every gap breaks that chain.

The instinct is to fix the transport. The short version: every road on that side closes. The longer version is worth walking through, because each fix you will reach for trades against the one thing you actually need: intact video. Reliable protocols like TCP or SRT recover lost packets, but they pay for it in latency, and on a variable link, packet loss becomes unavoidable the moment you impose any latency budget; below that budget the protocol simply discards the late packets, and you are back to gaps, now with visible glitches. Forward error correction, sending extra data in advance so losses can be repaired, helps until loss exceeds the fixed redundancy it carries, and it costs you permanent bandwidth overhead either way; it shines on protected networks, not on a drone climbing through changing conditions. Turning the bitrate down stops you dropping frames and starts you degrading them, and a smeared frame fails your analysis just as surely as a missing one. A better radio raises the ceiling; we tried that too: an outdoor 5G router, weatherproofed for the field, on the carrier whose coverage data and tower map looked best for the site. The ceiling was higher. It still was not enough: high-definition video can saturate the uplink, and altitude and motion swing the link too fast for any one setting to hold.

There is a single sentence underneath all of it. On a moving wireless link, you can pick two of reliability, fidelity, and low latency; the analysis needs all three. The trade-off is not a tuning problem with a setting you have not found yet. The link itself imposes it.

Pick two of reliability, fidelity, and low latency. The analysis needs all three; a moving link cannot hold them.

The sharp reader has already spotted the escape hatch: it is not real time anyway, so just record the full video onboard, upload the complete file afterward, and process it in the cloud. It sounds like a way out, and it does avoid dropped frames. But look at what it concedes and what it costs. It concedes that analyzing the live stream was never viable, which was the entire question. And it does not scale: full-resolution video saturates the same uplink you were trying to dodge, while storage and transfer balloon across a fleet or a long mission. It also cannot feed a live view, if you wanted one. Record-and-upload is not a rescue of the streaming plan. It is a slower, heavier version of the real answer.

The real answer is architectural, and it begins by naming what actually broke. This was, at bottom, not a model fault. It was a systems fault: a hidden assumption that data can reliably reach the compute. The moment you see it that way, the fix inverts. Do not move the video to the compute. Move the compute to the video. Put the processing on the drone, run it on the video stream exactly as the camera hands it over, every frame present, and let the network carry only what tolerates the trip.

The inversion: every frame stays on board; only loss-tolerant copies cross the link.

That last move is the quiet part of good systems design: deciding what must be reliable and what is allowed to drop. Everything is stored on board, raw and results alike; what crosses the link is only a copy. The live view, one feed of the raw video and one with the results drawn on, rides a link that is allowed to lose a few: a lost frame does not matter, because the original is sitting on the drone. It is the same footage under a different reliability budget. And the budget, not the codec, is the design.

What trips teams here is not a lack of skill. It is the order of the questions. The team asked which model should we use when the binding question was where does compute run, and what has to cross the network reliably. The model was not where it broke. And this particular fault line runs deep: answering it can push you past software entirely, into tailored hardware, a compute module specified for the drone’s size, weight, and power. That is not a detour from the work. On problems like this, it is the work.

We think of these as fault lines: the places where a benchmark-grade model, which earns its score under curated, clean, well-connected conditions, cracks against a real deployment. There are five (we map them in The Five Fault Lines), and systems and hardware is the deepest, because it can take you below the model altogether. The teams that ship tend to be the ones who go looking for their fault lines before they choose a model, not after the demo.

So before you pick a model, find your fault lines. The first one to check is rarely the algorithm. It is usually the wire.