Space Debris; Image enhancement; Deep Learning; Object tracking; Space; Artificial Intelligence; IAC-22; A6; x71536
Abstract :
[en] Images generated in space with monocular camera payloads suffer degradations that hinder their utility in precision tracking applications including debris identification, removal, and in-orbit servicing. To address the substandard quality of images captured in space and make them more reliable in space object tracking applications, several Image Enhancement (IE) techniques are investigated in this work. In addition, two novel space IE methods were developed. The first method called REVEAL, relies upon the application of more traditional image processing enhancement techniques and assumes a Retinex image formation model. A subsequent method, based on a UNet Deep Learning (DL) model was also developed. Image degradations addressed include blurring, exposure issues, poor contrast, and noise. The shortage of space-generated data suitable for supervised DL is also addressed. A visual comparison of both techniques developed was conducted and compared against the current state-of-the-art in DL-based IE methods relevant to images captured in space. It is determined in this work that both the REVEAL and the UNet-based DL solutions developed are well suited to correct for the degradations most often found in space images. In addition, it has been found that enhancing images in a pre-processing stage facilitates the subsequent extraction of object contours and metrics. By extracting information through image metrics, object properties such as size and orientation that enable more precise space object tracking may be more easily determined. Keywords: Deep Learning, Space, Image Enhancement, Space Debris
Research center :
University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > >
Disciplines :
Computer science
Author, co-author :
JAMROZIK, Michele Lynn ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > CVI2
GAUDILLIERE, Vincent ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > CVI2
MOHAMED ALI, Mohamed Adel ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > CVI2
AOUADA, Djamila ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > CVI2
External co-authors :
no
Language :
English
Title :
Image Enhancement for Space Surveillance and Tracking
Publication date :
2022
Event name :
International Astronautical Congress
Event organizer :
International Astronautical Federation
Event place :
Paris, France
Event date :
from 18-09-22 to 22-09-22
Audience :
International
Main work title :
Proceedings of the 73rd International Astronautical Congress
Jerry E. White Roger R. Bate Donald D. Mueller. Fundamentals of Astrodynamics. New York, USA: Dover Press, 1971.
Gonzalez and Woods. Digital Image Processing, 4th edition. Pearson, 2018.
Kaiming He, Jian Sun, and Xiaoou Tang. “Single Image Haze Removal Using Dark Channel Prior”. In: IEEE Transactions on Pattern Analysis and Machine Intelligence 33.12 (2011), pp. 2341-2353. DOI: 10.1109/TPAMI.2010.168.
Xuan Dong et al. “Fast efficient algorithm for enhancement of low lighting video”. In: Proceedings - IEEE International Conference on Multimedia and Expo. Vol. 7. Aug. 2011, pp. 1-6. DOI: 10.1109/ICME.2011.6012107.
John D. Austin and Stephen M. Pizer. “A Multiprocessor Adaptive Histogram Equalization Machine”. In: Information Processing in Medical Imaging. Boston, MA: Springer US, 1988, pp. 375-392. ISBN: 978-1-4615-7263-3. DOI: 10.1007/978-1-4615-7263-3_25. URL: https://doi.org/10.1007/978-1-4615-7263-3_25.
Karel Zuiderveld. “Contrast Limited Adaptive Histogram Equalization”. In: Graphics Gems IV. USA: Academic Press Professional, Inc., 1994, pp. 474-485. ISBN: 0123361559.
Brian Funt, Florian Ciurea, and John Mccann. “Retinex in Matlab”. In: Journal of Electronic Imaging. 2000, pp. 112-121.
Yonghua Zhang, Jiawan Zhang, and Xiaojie Guo. “Kindling the Darkness: A Practical Low-Light Image Enhancer”. In: Proceedings of the 27th ACM International Conference on Multimedia. MM'19. Nice, France: Association for Computing Machinery, 2019, pp. 1632-1640. ISBN: 9781450368896. DOI: 10.1145/3343031.3350926. URL: https://doi.org/10.1145/3343031.3350926.
David Libouga Li Gwet et al. “A Review on Image Segmentation Techniques and Performance Measures”. In: International Journal of Computer and Information Engineering 12.12 (2018), pp. 1107-1117. ISSN: eISSN: 1307-6892. URL: https://publications.waset.org/vol/144.
Kin Gwn Lore, Adedotun Akintayo, and Soumik Sarkar. “LLNet: A deep autoencoder approach to natural low-light image enhancement”. In: Pattern Recognition 61 (2017), pp. 650-662.
P. Maharjan et al. “Improving Extreme Low-Light Image Denoising via Residual Learning”. In: 2019 IEEE International Conference on Multimedia and Expo (ICME). July 2019, pp. 916-921. DOI: 10.1109/ICME.2019.00162.
Sean Moran et al. “DeepLPF: Deep Local Parametric Filters for Image Enhancement”. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 2020.
Vladimir Bychkovsky et al. “Learning Photographic Global Tonal Adjustment with a Database of Input/Output Image Pairs”. In: The Twenty-Fourth IEEE Conference on Computer Vision and Pattern Recognition. 2011.
Mahmoud Afifi et al. “Learning Multi-Scale Photo Exposure Correction”. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2021.
SMA Sharif et al. “A Two-stage Deep Network for High Dynamic Range Image Reconstruction”. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 2021.
Xiaodan Hu et al. “RUNet: A Robust UNet Architecture for Image Super-Resolution”. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops. June 2019.
Ben Moseley et al. Low-light image enhancement of permanently shadowed lunar regions with physics-based machine learning. eng. 2020.
Chen Chen et al. “Learning to See in the Dark”. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018, pp. 3291-3300. DOI: 10.1109/CVPR.2018.00347.
Feifan Lv, Yu Li, and Feng Lu. “Attention Guided Low-Light Image Enhancement with a Large Scale Low-Light Simulation Dataset”. en. In: arXiv:1908.00682 [cs, eess] (Mar. 2020). arXiv: 1908.00682 [cs, eess].
Feifan Lv, Y. Li, and Feng Lu. “Attention Guided Low-light Image Enhancement with a Large Scale Low-light Simulation Dataset”. In: arXiv: Image and Video Processing (2019).
Zhendong Wang et al. Uformer: A General U-Shaped Transformer for Image Restoration. 2021. arXiv: 2106.03106 [cs.CV].
Yonghua Zhang et al. “Beyond Brightening Low-light Images”. In: International Journal of Computer Vision 129 (2021), pp. 1013-1037. ISSN: 1573-1405. DOI: 10.1007/s11263-020-01407-x. URL: https://doi.org/10.1007/s11263-020-01407-x.
Bee Lim et al. “Enhanced Deep Residual Networks for Single Image Super-Resolution”. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops. July 2017.
Wenzhe Shi et al. “Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network”. In: CoRR abs/1609.05158 (2016). arXiv: 1609.05158. URL: http://arxiv.org/abs/1609.05158.
Xintao Wang et al. “ESRGAN: Enhanced super-resolution generative adversarial networks”. In: The European Conference on Computer Vision Workshops (ECCVW). Sept. 2018. URL: https://github.com/xinntao/ESRGAN.
Marc Bosch, Christopher M. Gifford, and Pedro A. Rodriguez. “Super-Resolution for Overhead Imagery Using DenseNets and Adversarial Learning”. In: CoRR abs/1711.10312 (2017). arXiv: 1711.10312. URL: http://arxiv.org/abs/1711.10312.
Chen Wei et al. “Deep Retinex Decomposition for Low-Light Enhancement”. In: British Machine Vision Conference. 2018.
Jiuxiang Gu et al. “Recent advances in convolutional neural networks”. In: Pattern Recognition 77 (2018), pp. 354-377. ISSN: 0031-3203. DOI: https://doi.org/10.1016/j.patcog.2017.10.013. URL: https://www.sciencedirect.com/science/article/pii/S0031320317304120.
Xinjie Li, Guojia Hou, and Kunqian Li. Enhancing Underwater Image via Adaptive Color and Contrast Enhancement, and Denoising. 2021. arXiv: 2104. 01073 [cs.CV].
Yudong Wang et al. “UIEC2̂-Net: CNN-based Underwater Image Enhancement Using Two Color Space”. In: CoRR abs/2103.07138 (2021). arXiv: 2103.07138. URL: https://arxiv.org/abs/2103.07138.
Hao-Hsiang Yang, Kuan-Chih Huang, and Wei-Ting Chen. LAFFNet: A Lightweight Adaptive Feature Fusion Network for Underwater Image Enhancement. 2021. arXiv: 2105.01299 [cs.CV].
Chongyi Li et al. “Lighting the Darkness in the Deep Learning Era”. In: arXiv:2104.10729 (2021).
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. “U-Net: Convolutional Networks for Biomedical Image Segmentation”. en. In: arXiv:1505.04597 [cs] (May 2015). arXiv: 1505.04597 [cs].
Kaiming He et al. “Deep Residual Learning for Image Recognition”. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016, pp. 770-778. DOI: 10.1109/CVPR.2016.90. URL: https://arxiv.org/pdf/1512.03385.pdf.
Wenzhe Shi et al. “Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network”. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 2016, pp. 1874-1883. DOI: 10.1109/CVPR.2016.207.
Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual Losses for Real-Time Style Transfer and Super-Resolution. 2016. arXiv: 1603.08155 [cs.CV].
Jeremy Howard and Sylvain Gugger. “fastai: A Layered API for Deep Learning”. In: CoRR abs/2002.04688 (2020). arXiv: 2002.04688. URL: https://arxiv.org/abs/2002.04688.
Sergey Ioffe and Christian Szegedy. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. 2015. arXiv: 1502. 03167 [cs.LG].
Diederik P. Kingma and Jimmy Ba. “Adam: A Method for Stochastic Optimization”. In: CoRR abs/1412.6980 (2015).
Mohamed Adel Musallam et al. “Spacecraft Recognition Leveraging Knowledge of Space Environment: Simulator, Dataset, Competition Design and Analysis”. In: 2021 IEEE International Conference on Image Processing Challenges (ICIPC). 2021, pp. 11-15. DOI: 10.1109/ICIPC53495.2021.9620184.
Pedro Proenca. URSO: Unreal Rendered Spacecrafts On-Orbit Datasets. July 2019. DOI: 10.5281/zenodo.3279632. URL: https://doi.org/10.5281/zenodo.3279632.
Alexander Buslaev et al. “Albumentations: Fast and Flexible Image Augmentations”. In: Information 11.2 (2020). ISSN: 2078-2489. DOI: 10.3390/info11020125. URL: https://www.mdpi.com/2078-2489/11/2/125.
