Angle-Aware Coverage Control for 3D Map Reconstruction

A 3D reconstruction is only as good as the images it is built from. A point on a surface seen from one direction reconstructs badly; seen from several, it reconstructs well. So the question for a drone team is not where should we fly but which viewpoints do we still need — and coverage control is a scalable way to answer it.

Angle-aware coverage control makes that literal. Instead of covering a 2D plane, the drones cover a five-dimensional virtual field: three spatial coordinates for the point being observed, plus the horizontal and vertical angle it is seen from. Every cell of that field carries an importance index, high while unobserved and decaying as cameras cover it from enough distinct directions. A QP-based controller constrains the decay rate of the resulting objective, so sampling performance is certified rather than hoped for.

This project is the thread of work extending that idea in three directions.

Making it work on large fields

The original controller has a scaling flaw. Its objective never evaluates how far an unobserved point is, so distant points get no priority, and the field ends up speckled with small unobserved patches that the drone crosses the whole map to collect one at a time. Barely visible on a testbed; dominant on a real site.

The fix was to leave the objective alone and change the nominal input of the QP instead. A classical Voronoi move-to-centroid term pulls each drone toward the weighted centroid of its own cell, so distance enters through where a drone wants to go, while the angle-aware constraint still governs what it may do.

Single drone on a large field: ~1300 s down to ~900 s. Three drones: 470 s down to 330 s. Validated in ROS and flown on the Tokyo Tech Robot Zoo Sky testbed with Parrot Bebop 2 drones under OptiTrack motion capture.

Making the camera move

In the original formulation the cameras point straight down and never move, so the only way to change a viewing angle is to fly somewhere else — leaving the cheapest degree of freedom unused.

Putting the camera on a gimbal and controlling it gives each drone a four-dimensional state: two position coordinates plus two gimbal angles. Covering a point becomes two conditions rather than one — it has to be in frame, and seen from a direction it still needs.

The cost is computational. With a steerable camera the 5D field no longer collapses onto a 2D one, so the full problem has to be solved directly. JAX, with JIT compilation and GPU acceleration, brought the per-step time from 1206 ms to 22 ms — the difference between an idea and a controller.

Making the mission persist

A field worth reconstructing takes longer to cover than a battery lasts, so charging belongs inside the control problem. Fixing which station each drone returns to wastes the flight time the constraint was meant to protect; assigning stations dynamically through ADMM does not.

Where it goes next

The natural continuation is closing the loop with the reconstruction itself — using the evolving 3D map as a feedback signal to tell the fleet where it is still unresolved. That became a separate project: Multi-Drone Image Sampling with Online Map Feedback.

Applications

Precision agriculture and crop inspection, rapid mapping of disaster zones, construction-site progress monitoring, and urban infrastructure survey — anywhere the reconstruction has to be good enough to measure from, not just to look at.

Video