Airborne Laser Scanning; City Modelling; Deep Learning; LiDAR; Machine Learning; Scene Understanding
Abstract :
[en] Semantic segmentation of point clouds is indispensable for 3D scene understanding. Point clouds have credibility for capturing geometry of objects including shape, size, and orientation. Deep learning (DL) has been recognized as the most successful approach for image semantic segmentation. Applied to point clouds, performance of the many DL algorithms degrades, because point clouds are often sparse and have irregular data format. As a result, point clouds are regularly first transformed into voxel grids or image collections. PointNet was the first promising algorithm that feeds point clouds directly into the DL architecture. Although PointNet achieved remarkable performance on indoor point clouds, its performance has not been extensively studied in large-scale outdoor point clouds. So far, we know, no study on large-scale aerial point clouds investigates the sensitivity of the hyper-parameters used in the PointNet. This paper evaluates PointNet’s performance for semantic segmentation through three large-scale Airborne Laser Scanning (ALS) point clouds of urban environments. Reported results show that PointNet has potential in large-scale outdoor scene semantic segmentation. A remarkable limitation of PointNet is that it does not consider local structure induced by the metric space made by its local neighbors. Experiments exhibit PointNet is expressively sensitive to the hyper-parameters like batch-size, block partition and the number of points in a block. For an ALS dataset, we get significant difference between overall accuracies of 67.5% and 72.8%, for the block sizes of 5m×5m and 10m×10m, respectively. Results also discover that the performance of PointNet depends on the selection of input vectors.
Disciplines :
Physical, chemical, mathematical & earth Sciences: Multidisciplinary, general & others
Author, co-author :
Nurunnabi, Abdul Awal Md ; University of Luxembourg > Faculty of Science, Technology and Medicine (FSTM) > Department of Engineering (DoE)
Teferle, Felix Norman ; University of Luxembourg > Faculty of Science, Technology and Medicine (FSTM) > Department of Engineering (DoE)
Li, J.
Lindenbergh, R. C.
PARVAZ, Shahoriar ; University of Luxembourg > Faculty of Science, Technology and Medicine (FSTM) > Department of Engineering (DoE)
External co-authors :
yes
Language :
English
Title :
INVESTIGATION OF POINTNET FOR SEMANTIC SEGMENTATION OF LARGE-SCALE OUTDOOR POINT CLOUDS
Publication date :
23 December 2021
Event name :
6th International Conference on Smart City Applications
Event organizer :
ISPRS
Event place :
Safranbolu, Turkey
Event date :
27- 29 October, 2021
Event number :
6
Audience :
International
Journal title :
International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences
Achlioptas, P., Diamanti, O., Mitliagkas, I., Guibas, L., 2018. Learning representations and generative models for 3D point clouds. Intl. Conf. Machine Learning, PMLR, 40 ?49.
Agoub, A., Schmidt, V., Kada, M., 2019. Generating 3D city models based on the semantic segmentation of lidar data using convolutional neural networks. ISPRS Ann. of Photogramm. Remote Sens. Spat. Info. Sci., 4.
Boulch, A., 2020. ConvPoint: Continuous convolutions for point cloud processing. Computers & Graphics, 88: 24?34.
Chehata, N., Guo, L., Mallet, C., 2009. Airborne lidar feature selection for urban classification using random forests., ISPRS Archive, XXXVIII-3(W8).
Chen, L., Wang, Q., Lu, X., Cao, D., Wang, F-Y., 2019. Learning driving models from parallel end-To-end driving data set. Proc. of the IEEE, 2019, 108(2): 262?273.
Gong, Z., et al., 2020. A frustum-based probabilistic framework for 3D object detection by fusion of LiDAR and camera data. ISPRS J. Photogramm. Remote Sens., 159: 90?100.
Guo, Z., et al., 2018. Semantic segmentation for urban planning maps based on U-Net. IGARSS, 6187?6190.
Guo, Y., Wang, H., Hu, Q., Liu, H., Liu, L., Bennamoun, 2020. Deep learning for 3D point clouds A survey.IEEE TPAMI,1?27.
Hackel, T., Savinov, N., Ladicky, L., Wegner, J. D., Schindler, K., Pollefeys, M., 2017. Semantic3D.Net: A new large-scale point cloud classification benchmark. arXiv:1704.03847.
Han, X., Dong, Z., Yang, B., 2021. A point-based deep learning network for semantic segmentation of MLS point clouds. ISPRS J. Photogramm. Remote Sens., 175:199 ?214.
Hu, Q., et al., 2020. RandLA-Net: Efficient semantic segmentation of large-scale point clouds. CVPR, 11108?11117.
Ioffe, S., Szegedy, C., 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. ICML, 448?456.
Isenburg, M., 2014. LAStools-efficient LiDAR processing software, http://rapidlasso.com/LAStools
Jaderberg, M., Simonyan, K., Zisserman, A., Kavukcuoglu, K., 2015. Spatial transformer networks. ArXiv:1506.02025.
Jing, Z,, Guan, H., Zhao, P., Li, D., Yu, Y., Zang, Y., Wang, H., Li, J., 2021. Multispectral LiDAR point cloud classification using SE-PointNet++, Remote Sensing, 13, 2516.
Kang, Z., Li, N., 2019. PyramNet: Point cloud pyramid attention network and graph embedding module for classification and segmentation. Australian Journal of Intelligent Information Processing Systems, 16 (2): 35?43.
Kingma, D. P., Ba, J., 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
Ko, Y., Lee, S-H., 2020. Novel method of semantic segmentation applicable to augmented reality. Sensors 20(6): 1737.
Krizhevsky, A., Sutskever, I., Hinton, G. E., 2012. Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process Syst, 25: 1097?1105.
Landrieu, L., Simonovsky, M., 2018. Large-scale point cloud semantic segmentation with superpoint graphs. IEEE CVPR,4558?4567.
LeCun, Y., et al., 1989. Backpropagation applied to handwritten zip code recognition. Neural Comput., 1(4): 541?551.
Li, G., Yang, Y., Qu, X., 2019. Deep learning approaches on pedestrian detection in hazy weather. IEEE Trans. Ind. Electron., 67(10): 8889?8899.
Li, W., Luo, Z., Xiao, Z., Chen, Y., Wang, C., Li, J., 2021. A GCN-based method for extracting power lines and pylons from airborne LiDAR data, IEEE Trans. Geosci. Remote Sens.
Ma, L., Li, Y., Li, J., Tan, W., Yu, Y., Chapman, M., 2020. Multi-scale point-wise convolutional neural networks for 3D object segmentation from LiDAR point clouds in large-scale urban environments, IEEE Trans. Intell. Transp. Syst.
Minaee, S., Boykov, Y., Porikli, F., Plaza, A., Kehtarnavaz, N., Terzopoulos, D., 2021. Image segmentation using deep learning: A survey. IEEE TPAMI, doi: 10.1109/TPAMI.2021.3059968.
Nair, V., Hinton, G., 2010. Rectified linear units improve restricted Boltzmann machines. ICML.
Niemeyer, J., Rottensteiner, F., Soergel, U., 2014. Contextual classification of lidar data and building object detection in urban areas. ISPRS J. Photogramm. Remote Sens., 87:152?165.
Nurunnabi, A., Belton, D., West, G., 2014. Robust statistical approaches for local planar surface fitting in 3D laser scanning data. ISPRS J. Photogramm. Remote Sens., 96: 106 ?122.
Nurunnabi, A., West, G., Belton, D., 2015. Outlier detection and robust normal-curvature estimation in mobile laser scanning 3D point cloud data. Pattern Recognit., 48(4):1404?1419.
Nurunnabi, A., Belton, D.,West, G., 2016. Robust segmentation for large volumes of laser scanning three-dimensional point cloud data.IEEE Trans. Geosci. Remote Sens.,54(8 ):4790-4805.
Nurunnabi, A.,Teferle, F. N., Li, J., Lindenbergh, R., Hunegnaw, A., 2021. An efficient deep learning approach for ground point filtering in aerial laser scanning point clouds. Int. Arch. of the Photogramm. Remote Sens. and Spat. Info. Sci., 24:1?8, XXIV-ISPRS Congress, 5-9 July.
Qi, C. R., Su, H., Nießner, M., Dai, A., Yan, M., Guibas, L., 2016. Volumetric and multi-view CNNs for object classification on 3d data. IEEE CVPR.
Qi, C. R., Su, H., Mo, K., Guibas, L. J., 2017a. PointNet: Deep learning on point sets for 3d classification and segmentation. IEEE CVPR, 652?660.
Qi, C. R., Yi, L., Su, H., Guibas, L. J., 2017b. PointNet++: Deep hierarchical feature learning on point sets in a metric space. arXiv preprint arXiv:1706.02413.
Rethage, D., Wald, J., Sturm, J., Navab, N., Tombari, F., 2018. Fully-convolutional point networks for large-scale point clouds. ECCV, 596 ?611.
Romero-Jarén, R., Arranz, J. J.,2021. Automatic segmentation and classification of BIM elements from point clouds. Autom. Constr., 124:103576, 1?17.
Su, H., Maji, S., Kalogerakis, E., Learned-Miller, E. G., 2015. Multi-view convolutional neural networks for 3d shape recognition. ICCV.
Su, H., et al., 2018. SPLATNet: Sparse lattice networks for point cloud processing. IEEE CVPR, 2530?2539.
Sun, X., Lian, Z., Xiao, J., 2019. SRINet: Learning strictly rotation-invariant representations for point cloud classification and segmentation. ACM Int. Conf. Multimedia, 980?988.
Thomas, H., Qi, C. R., Deschaud, J-E., Marcotegui, B., Goulette, F., Guibas. L. J., 2019. KPConv: Flexible and deformable convolution for point clouds. IEEE ICCV, 6411?6420.
Varney, N., Asari, V. K., Graehling, Q., 2020. DALES: A large-scale aerial LiDAR data set for semantic segmentation. IEEE CVPR Workshops, 186 ?187.
Wang, Y., Sun, Y., Liu, Z., Sarma, S. E., Bronstein, M. M., Solomon, J. M., 2019. Dynamic graph CNN for learning on point clouds. ACM Transactions On Graphics, 38(5):1?12.
Yang, B., Wang, S., Markham, A., Trigoni, N., 2020. Robust attentional aggregation of deep feature sets for multi-view 3D reconstruction. IJCV, 128 (1): 53?73.
Yu, D., Ji, S., Liu, Wei, S., 2021. Automatic 3D building reconstruction from multi-view aerial images with deep learning. ISPRS J. Photogramm. Remote Sens., 171: 155 ?170.
Zaheer, M., Kottur, S., Ravanbakhsh, S., Poczos, B., Salakhutdinov, R., Smola, A., 2017. Deep sets, arXiv preprint arXiv:1703.06114.
Zhang, J., Lin, X., Ning, X., 2013. SVM-based classification of segmented airborne LiDAR point clouds in urban areas. Remote Sensing, 5(8): 3749?3775.
Zhang, L., Li, Z., Li, A., Liu, F., 2018. Large-scale urban point cloud labeling and reconstruction. ISPRS J. Photogramm. Remote Sens., 138: 86 ?100.
Zhao, H., Jiang, L., Fu, C-W., Jia, J., 2019. PointWeb: Enhancing local neighborhood features for point cloud processing. IEEE CVPR, 5565 ? 5573.
Zhang,W., Xiao, C., 2019. PCAN: 3D attention map learning using contextual information for point cloud-based retrieval. IEEE CVPR, 12436?12445.
