Semantic Knowledge Distillation for Onboard Satellite Earth Observation Image Classification

This study introduces a dynamic weighting knowledge distillation (KD) framework for efficient Earth observation (EO) image classification (IC) in resource-constrained environments. By leveraging EfficientViT and MobileViT as teacher models, this approach enables lightweight student models, specifically ResNet8 and ResNet16, to achieve over 90% accuracy, precision, and recall, meeting the confidence thresholds required for reliable classification. Unlike traditional KD with fixed weights, our dynamic weighting mechanism adjusts based on each teachers confidence, allowing the student model to prioritize more reliable knowledge sources. ResNet8, in particular, achieves substantial efficiency gains, with 97.5% fewer parameters, 96.7% fewer FLOPs, 86.2% lower power consumption, and 63.5% faster inference time compared to MobileViT. This significant reduction in complexity and resource demand makes ResNet8 an ideal choice for EO tasks, balancing high performance with practical deployment requirements. This confidence-driven, adaptable KD strategy demonstrates the potential of dynamic knowledge distillation to deliver high-performing, resource-efficient models for satellite-based EO applications. Reproducible codes are available from our shared Github repository.