A Map-Based Localization System for Ingenuity Using Deep Image Matching

The success of NASA’s Mars Helicopter Ingenuity has paved the way for aerial planetary exploration with future mission concepts that will require advanced autonomous capabilities to enable long-range navigation. In the absence of a Global Navigation Satellite System (GNSS) on Mars, a critical capability is localization within the global frame to eliminate pose estimation drift, which typically involves registering onboard images to orbital maps—e.g., derived from High-Resolution Imaging Science Experiment (HiRISE) data. However, the registration process poses several challenges, including texture-less terrain, illumination variations, and most relevant to Ingenuity, a large resolution difference between low-altitude observations and HiRISE. With current registration methods using template-matching and hand-crafted features struggling under the aforementioned challenges, we turn our attention to deep learning-based image matchers that have shown impressive generalization potential, but failed to be widely adopted for space applications due to the lack of large-scale annotated datasets for training. In this article, we present a map-based localization (MbL) system for Ingenuity that incorporates a state-of-the-art deep image matcher model. We justify the feasibility of this approach for future missions by demonstrating a training strategy that: 1) rapidly adapts the deep image matcher in a self-supervised manner using a minimal amount of Ingenuity navigation images; 2) generalizes to previously unseen flights; and 3) is robust to the large resolution difference and outperforms prior template and hand-crafted registration methods in terms of localization accuracy.