Impact of Real-time Map Feedback on Coordinated Image Sampling for 3D Reconstruction

Sampling images efficiently from diverse viewing angles is what determines the quality of a reconstructed 3D map, and coverage control is a natural fit for coordinating that across a drone team. With real-time reconstruction now practical, the map can be rebuilt continuously during the flight — which means it can be fed straight back into motion control.

This paper asks what that feedback is actually worth.

Approach

The mission is posed as an angle-aware coverage control problem: drones capture the field of interest from multiple angles rather than merely passing over it. Images are processed in real time by NeuralRecon to produce an evolving 3D mesh, and mesh changes across the field update the coverage importance index as the map develops.

A QP-based controller certifies sampling performance by constraining the decay rate of the objective function, so the guarantee holds even as the importance index keeps shifting underneath it.

Results

Simulations in Unity and ROS2 show the feedback-driven approach producing a more complete and more accurate 3D map than the equivalent method without map feedback.

This conference paper is extended by the journal version, Coverage-Recon, which adds multi-drone coordination, real-world experiments and a second feedback formulation.

Media: video · project page · PDF