Lightweight palm and finger tracking for real-time 3D gesture control

Hackenberg, Georg; McCall, Roderick; Broll, Wolfgang

Download

Paper published in a book (Scientific congresses, symposiums and conference proceedings)

Lightweight palm and finger tracking for real-time 3D gesture control

Hackenberg, Georg; McCall, Roderick; Broll, Wolfgang

2011 • In IEEE Conference on Virtual Reality 2011

Peer reviewed

Permalink
https://hdl.handle.net/10993/5656

Files (1)Send to Details Statistics Bibliography Similar publications

Files

Full Text

vr2011paper.pdf

Publisher postprint (4.81 MB)

Download

All documents in ORBilu are protected by a user license.

Send to

RIS BibTex APA Chicago Permalink X Linkedin

Details

Abstract :

[en] We present a novel technique implementing barehanded interaction with virtual 3D content by employing a time-of-flight camera. The system improves on existing 3D multi-touch systems by working regardless of lighting conditions and supplying a working volume large enough for multiple users. Previous systems were limited either by environmental requirements, working volume, or computational resources necessary for realtime operation. By employing a time-of-flight camera, the system is capable of reliably recognizing gestures at the finger level in real-time at more than 50 fps with commodity computer hardware using our newly developed precision hand and finger-tracking algorithm. Building on this algorithm, the system performs gesture recognition with simple constraint modeling over statistical aggregations of the hand appearances in a working volume of more than 8 cubic meters. Two iterations of user tests were performed on a prototype system, demonstrating the feasibility and usability of the approach as well as providing first insights regarding the acceptance of true barehanded touch-based 3D interaction.

Disciplines :

Computer science

Identifiers :

UNILU:UL-CONFERENCE-2011-206

Author, co-author :

Hackenberg, Georg; Fraunhofer FIT

McCall, Roderick ; Fraunhofer FIT

Broll, Wolfgang; Fraunhofer FIT

Language :

English

Title :

Lightweight palm and finger tracking for real-time 3D gesture control

Publication date :

2011

Event name :

Virtual Reality

Event place :

Singapore, Singapore

Event date :

19-23 March 2011

Main work title :

IEEE Conference on Virtual Reality 2011

Publisher :

IEEE

ISBN/EAN :

978-1-4577-0039-2

Pages :

19-26

Peer reviewed :

Peer reviewed

Commentary :

IEEE VR 2011

Available on ORBilu :

since 06 September 2013

Statistics

Number of views

49 (3 by Unilu)

Number of downloads

706 (0 by Unilu)

More statistics

Scopus citations^®

Scopus citations^®
without self-citations

Bibliography

K. Abe, H. Saito and S. Ozawa. "3-D drawing system via hand motion recognition from two cameras," in Proc. of IEEE SMC, 2000, pp. 840-845 vol. 2. (Pubitemid 32069140)
A. Argyros and M. Lourakis. "Tracking multiple colored blobs with a moving camera," in Proc. of IEEE CVPR, 2005, 1178 vol. 2.
K. Barhate et al. "Robust shape based two hand tracker," in Proc. Of IEEE ICIP, 2004, pp. 1017-1020. (Pubitemid 40820209)
Q. Delamarre and O. Faugeras. "Finding pose of hand in video images: a stereo-based approach," in Proc. of IEEE FG, 1998, pp. 585-590.
M. Donoser and H. Bischof. "Real time appearance based hand tracking." in Proc. of ICPR, 2008, pp. 1-4.
A. El-Sawah, C. Joslin, N. Georganas and E. Petriu. "A Framework for 3D Hand Tracking and Gesture Recognition using Elements of Genetic Programming," in Proc. of IEEE CRV, 2007, pp. 495-502. (Pubitemid 47432508)
S. Ghobadi et al. "Real Time Hand Based Robot Control Using 2D/3D Images," in Proc. of ISVC, 2008, pp. 307-316.
J. Gould and C. Lewis. "Designing for usability: key principles and what designers think." CACM, vol. 28, no. 3, pp. 300-311, 1985. (Pubitemid 15599384)
T. Gumpp, P. Azad, K. Welke, E. Oztop, R. Dillmann and G. Cheng. "Unconstrained Real-time Markerless Hand Tracking for Humanoid Interaction," in Proc. of IEEE Humanoids, 2006, pp. 88-93.
S. Guomundsson et al. "Model-Based Hand Gesture Tracking in ToF Image Sequences," in Proc. of AMDO, 2010, pp. 118-127.
C. Keskin, A. Erkan and L. Akarun. "Real time hand tracking and 3d gesture recognition for interactive interfaces using hmm," in Proc. Of Int. Conf. on Artificial Neural Networks, 2003.
H. Kim, K. Kwak, S. Chi and Y. Cho. "AR-KLT based Hand Tracking," in Proc. of IEEE RO-MAN, 2006, pp. 611-616.
M. Kolsch and M. Turk. "Robust hand detection," in Proc. of IEEE FG, 2004, pp. 614-619.
M. Kolsch and M. Turk. "Hand tracking with Flocks of Features," in Proc. of IEEE CVPR, 2005, p. 1187 vol. 2.
R. Lange and P. Seitz. "Solid-state time-of-flight range camera." IEEE J. of Quantum Electronics, vol. 37, no. 3, pp. 390-397, 2001. (Pubitemid 32286777)
J. Letessier and F. Bérard. "Visual tracking of bare fingers for interactive surfaces," in Proc. of ACM UIST, 2004, pp. 119-122. (Pubitemid 40787672)
W. Liang, Y. Jia, F. Sun, B. Ning, T. Liu and X. Wu. "Visual Hand Tracking Using MDSA Method, "in Proc. of IMACS Multiconf. On Computational Eng. in Syst. Applications, 2006, pp. 255-259.
Y. Liu and Y. Jia. "A Robust Hand Tracking and Gesture Recognition Method for Wearable Visual Interfaces and Its Applications," in Proc. of IEEE ICIG, 2004, pp. 472-475. (Pubitemid 40498129)
S. Lu, D. Metaxas, D. Samaras and J. Oliensis. "Using multiple cues for hand tracking and model refinement in Proc. of IEEE CVPR, 2003, pp. 443-450 vol. 2.
B. Lucas and T. Kanade. "An iterative image registration technique with an application to stereo vision," in Proc. of IJCAI, 1981, pp. 674-679. (Pubitemid 14478744)
J. Matas, O. Chum, M. Urban and T. Pajdla. "Robust wide baseline stereo from maximally stable external regions, "in Proc. of British Machine Vision Conference, 2002, pp. 384-393.
Z. Mo, J. Lewis and U. Neumann. "SmartCanvas: a gesture-driven intelligent drawing desk system," in Proc. of ACM IUI, 2005, pp. 239-243.
J. Nielsen. "The usability engineering life cycle." IEEE Computer, vol. 25, no. 3, pp. 12-22, 1992.
P. Peltonen et al. "Extending large-scale event participation with user-created mobile media on a public display," in Proc. of ACM MUM. 2007, pp. 131-138.
R. Rosales, V. Athitsos, L. Sigal and S. Sclaroff. "3D hand pose reconstruction using specialized mappings," in Proc. of IEEE ICCV, 2001, pp. 378-385 vol. 1.
E. Rosten, R. Porter and T. Drummond. "Faster and Better: A Machine Learning Approach to Corner Detection." IEEE Trans. Pattern Anal. Mach. Intelligence, vol. 32, no. 1, pp. 105-119, 2010.
Y. Sato, M. Saito and H. Koike. "Real-time input of 3D pose and gestures of a user's hand and its applications for HCI, " in Proc. Of IEEE VR, 2001, pp. 79-86. (Pubitemid 32334735)
J. Segen and S. Kumar. "Gesture VR: vision-based 3D hand interface for spatial interaction," in Proc. of ACM MULTIMEDIA, 1998, pp. 455-464.
E. Ueda, Y. Matsumoto, M. Imai and T. Ogasawara, "A hand-pose estimation for vision-based human interfaces." IEEE Trans. Ind. Electronics, vol. 50, no. 4, pp. 676-684, 2003.
J. Wall. "Demo I Microsoft Surface and the Single View Platform," in Proc. of Int. Symp. on Collaborative Technologies and Systems, 2009, pp. xxxi-xxxii.
X. Wang, X. Zhang and G. Dai. "Tracking of deformable human hand in real time as continuous input for gesture-based interaction," in Proc. of ACM IUI, 2007, pp. 235-242. (Pubitemid 47464009)