MANTLE: A Framework for Adaptive In-Situ Planetary Perception Using a Modular Uplink Principle

Durai, Pranav; Doran, Gary

MANTLE: A Framework for Adaptive In-Situ Planetary Perception Using a Modular Uplink Principle

Pranav Durai¹, Gary Doran²

¹Stanford Center for Innovation in In Vivo Imaging, Stanford University School of Medicine
²Jet Propulsion Laboratory, California Institute of Technology

Paper Supplementary Code HiRISE Landform Dataset MSL Boulder Dataset

Proposed MANTLE architecture: A shared frozen DINOv2 ViT-S/14 backbone (snowflake) feeds two independently trained task-specific heads (flames denote trainable modules). Orbital HiRISE tiles are resized to 224 × 224 (16 × 16 patch tokens) for the terrain classification head, while ground-level MSL images are resized to 784 × 784 (56 × 56 patch tokens) for the boulder segmentation head. Each head is trained separately against the frozen backbone, so new heads can be added without retraining or modifying the backbone.

Abstract

Planetary surface exploration missions rely increasingly on autonomous robotic platforms capable of interpreting complex terrain to ensure safe navigation, enable targeted science, and improve operational efficiency, as demonstrated across past Mars missions from Viking through Perseverance. Among the key perception capabilities, landform classification provides contextual information for landing site selection and scientific analysis, while boulder segmentation supports hazard assessment and path planning.

This paper presents MANTLE, a multitask adaptive network for terrain and landform extraction. The model uses a shared DINOv2 backbone for high-level feature extraction with task-specific heads: a classification head for large-scale landform classification, and a segmentation head for pixel-wise boulder localization, each trained on curated datasets built respectively from HiRISE orbital imagery and MSL surface-level imagery. The classification head achieved a test accuracy of 92.56% across seven Martian terrain classes, while the segmentation head achieved a validation IoU of 0.753 and showed strong cross-sol generalization on a held-out test set from previously unseen rover traverses.

A key advantage of MANTLE is its modular, extensible design, formalized here as the Modular Uplink Principle: only a shared, frozen backbone needs to remain onboard, while subsequent perception capabilities are trained on Earth as lightweight task-specific heads and uplinked without retraining the full model. This work demonstrates two such high-impact capabilities, terrain classification and boulder segmentation, as an initial realization of a framework built to support many more over a mission's lifetime. With this foundation, future explorers need not arrive on Mars fully formed, but can continue to learn, adapt, and grow more capable with every uplink.

Terrain Classification Capability

Evaluation of the MANTLE classification head across seven distinct Martian landform classes from HiRISE orbital imagery (achieving 92.56% test accuracy).

Bright Dune

Crater

Dark Dune

Impact Ejecta

Slope Streak

Spider

Swiss Cheese

Boulder Segmentation Capability

Evaluation of the MANTLE boulder segmentation head on unseen MSL Martian surface test imagery (achieving 0.753 validation IoU and strong cross-sol generalization).

MANTLE Boulder Segmentation Heatmap Test 09

High IoU example (IoU = 0.90)

MANTLE Boulder Segmentation Heatmap Test 08

High-mid IoU example (IoU = 0.80)

MANTLE Boulder Segmentation Heatmap Test 07

Mid-range IoU example (IoU = 0.70)

MANTLE Boulder Segmentation Heatmap Test 05

Low IoU example (IoU = 0.50), dominated by small rocks and pebbles

Modular Uplink Principle

The Modular Uplink Principle. New perception capabilities are added to a deployed rover in three steps: (1) a task-specific head is trained and validated on Earth against a copy of the frozen, flight-qualified backbone, yielding a lightweight (KB–MB scale) module; (2) the trained head is uplinked via the Deep Space Network (DSN) and relayed to the surface by a Mars relay orbiter; and (3) the head executes onboard as a plug-in to the existing frozen backbone alongside previously deployed heads, enabling a new capability without retraining or modifying the backbone or onboard flight software.

Citation

Paper

@misc{durai2026mantleframeworkadaptiveinsitu,
      title={MANTLE: A Framework for Adaptive In-Situ Planetary Perception Using a Modular Uplink Principle}, 
      author={Pranav Durai and Gary Doran},
      year={2026},
      eprint={2608.28724},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2608.28724}, 
}

HiRISE Landform Classification Dataset

@dataset{durai_2026_21300384,
  author       = {Durai, Pranav and
                  Doran, Gary},
  title        = {HiRISE Landform Classification Dataset},
  month        = jul,
  year         = 2026,
  publisher    = {Zenodo},
  doi          = {10.5281/zenodo.21300384},
  url          = {https://doi.org/10.5281/zenodo.21300384},
}

MSL Boulder Segmentation Dataset

@dataset{durai_2026_21313774,
  author       = {Durai, Pranav and
                  Doran, Gary},
  title        = {MSL Boulder Segmentation Dataset},
  month        = jul,
  year         = 2026,
  publisher    = {Zenodo},
  doi          = {10.5281/zenodo.21313774},
  url          = {https://doi.org/10.5281/zenodo.21313774},
}