3DBodyTex; Human body modelling; 3D Body fitting; 3D body scans; Dataset
Résumé :
[en] In this paper, a dataset, named 3DBodyTex, of static 3D body scans with high-quality texture information is presented along with a fully automatic method for body model fitting to a 3D scan. 3D shape modelling is a fundamental area of computer vision that has a wide range of applications in the industry. It is becoming even more important as 3D sensing technologies are entering consumer devices such as smartphones. As the main output of these sensors is the 3D shape, many methods rely on this information alone. The 3D shape information is, however, very high dimensional and leads to models that must handle many degrees of freedom from limited information. Coupling texture and 3D shape alleviates this burden, as the texture of 3D objects is complementary to their shape. Unfortunately, high-quality texture content is lacking from commonly available datasets, and in particular in datasets of 3D body scans. The proposed 3DBodyTex dataset aims to fill this gap with hundreds of high-quality 3D body scans with high-resolution texture. Moreover, a novel fully automatic pipeline to fit a body model to a 3D scan is proposed. It includes a robust 3D landmark estimator that takes advantage of the high-resolution texture of 3DBodyTex. The pipeline is applied to the scans, and the results are reported and discussed, showcasing the diversity of the features in the dataset.
Centre de recherche :
Interdisciplinary Centre for Security, Reliability and Trust (SnT) > SIGCOM
Disciplines :
Sciences informatiques
Auteur, co-auteur :
SAINT, Alexandre Fabian A ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT)
AHMED, Eman ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT)
SHABAYEK, Abd El Rahman ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT)
CHERENKOVA, Kseniya ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT)
Gusev, Gleb
AOUADA, Djamila ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT)
OTTERSTEN, Björn ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT)
Co-auteurs externes :
no
Langue du document :
Anglais
Titre :
3DBodyTex: Textured 3D Body Dataset
Date de publication/diffusion :
2018
Nom de la manifestation :
3DV 2018, International Conference on 3DVision
Lieu de la manifestation :
Verona, Italie
Date de la manifestation :
from 05-09-2019 to 08-09-2018
Manifestation à portée :
International
Titre de l'ouvrage principal :
2018 Sixth International Conference on 3D Vision (3DV 2018)
H. Afzal, D. Aouada, D. Fofi, B. Mirbach, and B. Ottersten. Rgb-d multi-view system calibration for full 3d scene reconstruction. In Pattern Recognition (ICPR), 2014 22nd International Conference on, pages 2459-2464. IEEE, 2014. 2
S. Agarwal, K. Mierle, and Others. Ceres solver. http: //ceres-solver. org. 8
E. Ahmed, A. Saint, A. E. R. Shabayek, K. Cherenkova, G. Gusev, D. Aouada, and B. Ottersten. Deep learning advances on different 3d data representations: A survey, 2018. arXiv preprint. 2, 5
B. Allen, B. Curless, and Z. Popovíc. The space of human body shapes: reconstruction and parameterization from range scans. ACM transactions on graphics (TOG), 22(3):587-594, 2003. 2
B. Allen, B. Curless, Z. Popovíc, and A. Hertzmann. Learning a correlated model of identity and pose-dependent body shape variation for real-time synthesis. In Proceedings of the 2006 ACM SIGGRAPH/Eurographics symposium on Computer animation, pages 147-156. Eurographics Association, 2006. 1
D. Anguelov, P. Srinivasan, D. Koller, S. Thrun, J. Rodgers, and J. Davis. Scape: shape completion and animation of people. In ACM Transactions on Graphics (TOG), volume 24, pages 408-416. ACM, 2005. 1, 2, 4, 6, 7
F. Bogo, A. Kanazawa, C. Lassner, P. Gehler, J. Romero, and M. J. Black. Keep it smpl: Automatic estimation of 3d human pose and shape from a single image. In European Conference on Computer Vision, pages 561-578. Springer, 2016. 1
F. Bogo, J. Romero, M. Loper, and M. J. Black. Faust: Dataset and evaluation for 3d mesh registration. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3794-3801, 2014. 1, 2, 3, 4
F. Bogo, J. Romero, G. Pons-Moll, and M. J. Black. Dynamic FAUST: Registering human bodies in motion. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), July 2017. 2, 4
A. M. Bronstein, M. M. Bronstein, and R. Kimmel. Numerical geometry of non-rigid shapes. Springer Science & Business Media, 2008. 1, 2, 3
M. M. Bronstein, J. Bruna, Y. LeCun, A. Szlam, and P. Vandergheynst. Geometric deep learning: going beyond euclidean data. IEEE Signal Processing Magazine, 34(4):18-42, 2017. 2, 5
L. Campagnola, A. Klein, E. Larson, C. Rossant, and N. P. Rougier. VisPy: Harnessing The GPU For Fast, High-Level Visualization. In K. Huff and J. Bergstra, editors, Proceedings of the 14th Python in Science Conference, Austin, Texas, United States, July 2015. 8
Z. Cao, T. Simon, S.-E. Wei, and Y. Sheikh. Realtime multiperson 2d pose estimation using part affinity fields. In CVPR, 2017. 5, 6, 8
K. I. Chang, K. W. Bowyer, and P. J. Flynn. An evaluation of multimodal 2d+ 3d face biometrics. IEEE transactions on pattern analysis and machine intelligence, 27(4):619-624, 2005. 2
F. Garcia, D. Aouada, T. Solignac, B. Mirbach, and B. Ottersten. Real-time depth enhancement by fusion for rgb-d cameras. IET Computer Vision, 7(5):335-345, 2013. 1
P. Guan, A. Weiss, A. O. Balan, and M. J. Black. Estimating human shape and pose from a single image. In 2009 IEEE 12th International Conference on Computer Vision, pages 1381-1388. IEEE, 2009. 1
N. Hasler, H. Ackermann, B. Rosenhahn, T. Thormählen, and H.-P. Seidel. Multilinear pose and body shape estimation of dressed subjects from image sets. In Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on, pages 1823-1830. IEEE, 2010. 1
N. Hasler, C. Stoll, B. Rosenhahn, T. Thormählen, and H.-P. Seidel. Estimating body shape of dressed humans. Computers & Graphics, 33(3):211-216, 2009. 5
N. Hasler, C. Stoll, M. Sunkel, B. Rosenhahn, and H.-P. Seidel. A statistical model of human pose and body shape. In Computer Graphics Forum, volume 28, pages 337-346. Wiley Online Library, 2009. 1, 2, 3, 4
D. A. Hirshberg, M. Loper, E. Rachlin, and M. J. Black. Coregistration: Simultaneous alignment and modeling of articulated 3d shape. In European Conference on Computer Vision, pages 242-255. Springer, 2012. 5
A. Jain, T. Thormählen, H.-P. Seidel, and C. Theobalt. Moviereshape: Tracking and reshaping of humans in videos. In ACM Transactions on Graphics (TOG), volume 29, page 148. ACM, 2010. 1, 2
V. G. Kim, Y. Lipman, and T. Funkhouser. Blended intrinsic maps. In ACM Transactions on Graphics (TOG), volume 30, page 79. ACM, 2011. 3
T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pages 740-755. Springer, 2014. 5
M. Loper, N. Mahmood, J. Romero, G. Pons-Moll, and M. J. Black. Smpl: A skinned multi-person linear model. ACM Transactions on Graphics (TOG), 34(6):248, 2015. 1, 2
O. K. Oyedotun, G. Demisse, A. E. R. Shabayek, D. Aouada, and B. Ottersten. Facial expression recognition via joint deep learning of rgb-depth map latent representations. In 2017 IEEE International Conference on Computer Vision Workshop (ICCVW), 2017. 1
L. Pishchulin, S. Wuhrer, T. Helten, C. Theobalt, and B. Schiele. Building statistical shape spaces for 3d human modeling. Pattern Recognition, 67:276-286, 2017. 1, 2, 3, 4
G. Pons-Moll, J. Romero, N. Mahmood, and M. J. Black. Dyna: A model of dynamic human shape in motion. ACM Transactions on Graphics, (Proc. SIGGRAPH), 34(4):120:1-120:14, Aug. 2015. 2, 4, 5
R. Poppe. A survey on vision-based human action recognition. Image and vision computing, 28(6):976-990, 2010. 1
K. M. Robinette, H. Daanen, and E. Paquet. The caesar project: A 3-d surface anthropometry survey. In 3-D Digital Imaging and Modeling, 1999. Proceedings. Second International Conference on, pages 380-386. IEEE, 1999. 1, 2, 3, 4
E. Rodolà, S. Rota Bulo, T. Windheuser, M. Vestner, and D. Cremers. Dense non-rigid shape correspondence using random forests. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4177-4184, 2014. 3
A. Saint, A. E. R. Shabayek, D. Aouada, B. Ottersten, K. Cherenkova, and G. Gusev. Towards automatic human body model fitting to a 3d scan. In Proceedings of 3DBODY. TECH 2017-8th International Conference and Exhibition on 3D Body Scanning and Processing Technologies, Montreal QC, Canada, 11-12 Oct. 2017, pages 274-280. Hometrica Consulting, 2017. 5
A. E. R. Shabayek, D. Aouada, A. Saint, and B. Ottersten. Deformation transfer of 3d human shapes and poses on manifolds. In Image Processing (ICIP), 2017 IEEE International Conference on, pages 220-224. IEEE, 2017. 1, 5
T. Simon, H. Joo, I. Matthews, and Y. Sheikh. Hand keypoint detection in single images using multiview bootstrapping. In CVPR, 2017. 5, 6, 8
R. W. Sumner and J. Popovíc. Deformation transfer for triangle meshes. ACM Transactions on Graphics (TOG), 23(3):399-405, 2004. 1, 5
G. Varol, J. Romero, X. Martin, N. Mahmood, M. J. Black, I. Laptev, and C. Schmid. Learning from synthetic humans. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017), 2017. 4
S.-E. Wei, V. Ramakrishna, T. Kanade, and Y. Sheikh. Convolutional pose machines. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4724-4732, 2016. 1, 2, 5, 6, 8
A. Weiss, D. Hirshberg, and M. J. Black. Home 3d body scans from noisy image and range data. In 2011 International Conference on Computer Vision, pages 1951-1958. IEEE, 2011. 5
S. Wuhrer, L. Pishchulin, A. Brunton, C. Shu, and J. Lang. Estimation of human body shape and posture under clothing. Computer Vision and Image Understanding, 127:31-42, 2014. 5
Z. Xu, Q. Zhang, and S. Cheng. Multilevel active registration for kinect human body scans: from low quality to high quality. Multimedia Systems, pages 1-14, 2017. 1, 2, 3, 5
J. Yang, J.-S. Franco, F. Hétroy-Wheeler, and S. Wuhrer. Estimation of human body shape in motion with wide clothing. In European Conference on Computer Vision, pages 439-454. Springer, 2016. 2
Y. Yang, Y. Yu, Y. Zhou, S. Du, J. Davis, and R. Yang. Semantic parametric reshaping of human body models. In 2014 2nd International Conference on 3D Vision, volume 2, pages 41-48. IEEE, 2014. 1, 2, 3, 4
C. Zhang, S. Pujades, M. Black, and G. Pons-Moll. Detailed, accurate, human shape estimation from clothed 3d scan sequences. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), volume 2, 2017. 2
