Oopses and Ohs: An Eye-Tracking Study of Human Errors in Virtual Reality

NIKNAM, Sahar; SIULHIN, Vladyslav; BOTEV, Jean

doi:10.1109/VRW70859.2026.00021

Request a copy

Paper published in a book (Scientific congresses, symposiums and conference proceedings)

Oopses and Ohs: An Eye-Tracking Study of Human Errors in Virtual Reality

NIKNAM, Sahar; SIULHIN, Vladyslav; BOTEV, Jean

2026 • In Proceedings of the IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)

Peer reviewed Dataset

Permalink
https://hdl.handle.net/10993/67533

DOI
10.1109/VRW70859.2026.00021

Files (1)Send to Details Statistics Bibliography Similar publications

Files

Full Text

Niknam 2026 - Oopses and Ohs_ An Eye-Tracking Study of Human Errors in Virtual Reality.pdf

Publisher postprint (1.18 MB)

Request a copy

All documents in ORBilu are protected by a user license.

Send to

RIS BibTex APA Chicago Permalink X Linkedin

Details

Keywords :

human error; cognitive load; eye-tracking; virtual reality; biosignals; deep learning

Abstract :

[en] Human error remains a major source of accidents and casualties across domains, from manufacturing to healthcare. Building on Rasmussen’s Skill–Rule–Knowledge (SRK) model and Reason’s taxonomy, we investigate whether eye-tracking data can classify and predict cognitive error types. We conducted an 18-month longitudinal study in virtual reality (VR), collecting over 60 hours of eye-tracking data from two participants performing mental arithmetic. Samples were labeled as Correct, Missed, or Wrong and used to train an LSTM neural network. The model reliably separated correct from missed responses using pupil diameter, gaze dynamics, and blinks. However, wrong responses were frequently confused with correct ones, reflecting the challenge of detecting skill-based slips accompanied by false confidence. Predicting human error is especially critical when conventional practice is costly or risky, such as high-precision surgery or military operations. Our findings highlight the potential of eye-tracking data to advance the understanding, prediction, and prevention of human error.

Disciplines :

Computer science

Author, co-author :

NIKNAM, Sahar ; University of Luxembourg > Faculty of Science, Technology and Medicine (FSTM) > Department of Computer Science (DCS)

SIULHIN, Vladyslav ; University of Luxembourg > Luxembourg Centre for Contemporary and Digital History (C2DH) > Contemporary History of Luxembourg

BOTEV, Jean ; University of Luxembourg > Faculty of Science, Technology and Medicine (FSTM) > Department of Computer Science (DCS)

External co-authors :

Language :

English

Title :

Oopses and Ohs: An Eye-Tracking Study of Human Errors in Virtual Reality

Publication date :

2026

Event name :

The 33rd IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR)

Event place :

Daegu, South Korea

Event date :

21-25 March 2026

Audience :

International

Main work title :

Proceedings of the IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)

Publisher :

IEEE

Peer review/Selection committee :

Peer reviewed

Data Set :

https://zenodo.org/records/18376588

Longitudinal eye-tracking data collected from two participants engaged in a mental arithmetic task in virtual reality, spanning 18- and 6-month periods.

Available on ORBilu :

since 28 January 2026

Statistics

Number of views

172 (60 by Unilu)

Number of downloads

13 (5 by Unilu)

More statistics

Scopus citations^®

Scopus citations^®
without self-citations

OpenAlex citations

Bibliography

J. R. Anderson and C. J. Lebiere. The atomic components of thought. Psychology Press, 2014. doi: 10.4324/9781315805696
I. Andras, E. Mazzone, F. W. B. Van Leeuwen, G. De Naeyer, M. N. Van Oosterom, S. Beato, T. Buckle, S. O'Sullivan, P. J. Van Leeuwen, A. Beulens, N. Crisan, F. D'Hondt, P. Schatteman, H. Van Der Poel, P. Dell'Oglio, and A. Mottrie. Artificial intelligence and robotics: A combination that is changing the operating room. World journal of urology, 38(10):2359-2366, 2020. doi: 10.1007/s00345-019-03037-6
V. Anu, W. Hu, J. C. Carver, G. S. Walia, and G. Bradshaw. Development of a human error taxonomy for software requirements: A systematic literature review. Information and Software Technology, 103:112-124, 2018. doi: 10.1016/j.infsof.2018.06.011
M. L. Bolton, E. J. Bass, and R. I. Siminiceanu. Using formal methods to predict human error and system failures. In Proc. 2nd Int. Conf. appl. human factors ergonom, 2008.
J.W. Brascamp, B. Zhang, and V. Marshev. Fixational saccade inhibition and pupil dilation during self-paced limb movement preparation. Plos one, 20(10):e0335504, 2025. doi: 10.1371/journal.pone.0335504
S. A. Cardoso, J. Suyambu, J. Iqbal, D. C. C. Jaimes, A. Amin, J. T. Sikto, M. Valderrama, S. S. Aulakh, V. Ramana, B. Shaukat, et al. Exploring the role of simulation training in improving surgical skills among residents: A narrative review. Cureus, 15(9), 2023. doi: 10. 7759/cureus.44654
X. Chen, H. Wang, X. Tan, M. Duan, and C. Luo. Flight training and the anterior cingulate cortex. Scientific Reports, 14(1):1-6, 2024. doi: 10.1038/s41598-024-81892-z
R. J. Compton, D. Gearinger, H. Wild, D. Rette, E. C. Heaton, S. Histon, P. Thiel, and M. Jaskir. Simultaneous eeg and pupillary evidence for post-error arousal during a speeded performance task. European Journal of Neuroscience, 53(2):543-555, 2021. doi: 10.1111/EJN.14947
C. Danielmeier and M. Ullsperger. Post-error adjustments. Frontiers in psychology, 2:233, 2011. doi: 10.3389/fpsyg.2011.00233
S, . B. Demiral, P. Manza, E. Biesecker, C. Wiers, E. Shokri-Kojori, K. McPherson, E. Dennis, A. Johnson, D. Tomasi, G.-J. Wang, et al. Striatal d1 and d2 receptor availability are selectively associated with eye-blink rates after methylphenidate treatment. Communications Biology, 5(1):1015, 2022. doi: 10.1038/s42003-022-03979-5
C. Dewi, R. C. Chen, X. Jiang, and H. Yu. Adjusting eye aspect ratio for strong eye blink detection based on facial landmarks. PeerJ Computer Science, 8:e943, 2022. doi: 10.7717/peerj-cs.943
B. J. Dietvorst, J. P. Simmons, and C. Massey. Algorithm aversion: People erroneously avoid algorithms after seeing them err. Journal of experimental psychology: General, 144(1):114, 2015. doi: 10.1037/xge0000033.supp
M. R. Endsley and E. O. Kiris. The out-of-the-loop performance problem and level of control in automation. Human factors, 37(2):381-394, 1995. doi: 10.1518/001872095779064555
M. K. Holland and G. Tarlow. Blinking and thinking. Perceptual and motor skills, 41(2):403-406, 1975. doi: 10.2466/pms.1975.41.2.403
S. Jainta, M. Vernet, Q. Yang, and Z. Kapoula. The pupil reflects motor preparation for saccades-even before the eye starts to move. Frontiers in human neuroscience, 5:11905, 2011. doi: 10.3389/fnhum. 2011.00097
S. M. Jones-Jang and Y. J. Park. How do people react to ai failure? Automation bias, algorithmic aversion, and perceived controllability. Journal of Computer-Mediated Communication, 28(1):zmac029, 2023. doi: 10.1093/jcmc/zmac029
T. D. Kelley, D. J. Patton, and L. Allender. Predicting situation awareness errors using cognitive modeling. In Proceedings of the 2001 Human Computer Interaction International Conference, 2001.
J. H. Kim, Y.-A. Suh, and M.-S. Yim. An investigation of human error identification based on bio-monitoring system (eeg and ecg analysis). In International Conference on Applied Human Factors and Ergonomics, pp. 145-151. Springer, 2018. doi: 10.1007/978-3-319-94866-9 14
M. Kodappully, B. Srinivasan, and R. Srinivasan. Towards predicting human error: Eye gaze analysis for identification of cognitive steps performed by control room operators. Journal of Loss Prevention in the Process Industries, 42:35-46, 2016. doi: 10.1016/j.jlp.2015.07.001
A. Konert and T. Balcerzak. Military autonomous drones (uavs)-from fantasy to reality. legal and ethical implications. Transportation research procedia, 59:292-299, 2021. doi: 10.1016/j.trpro.2021.11.121
W. Kong, D. Zhou, Y. Du, Y. Zhou, and Y. Zhao. Reinforcement learning for multiaircraft autonomous air combat in multisensor ucav platform. IEEE Sensors Journal, 23(18):20596-20606, 2022. doi: 10. 1109/JSEN.2022.3220324
J. Y. Lee, N. De Jong, J. Donkers, H. Jarodzka, and J. J. G. Van Merriënboer. Measuring cognitive load in virtual reality training via pupillometry. IEEE Transactions on Learning Technologies, 17:704-710, 2023. doi: 10.1109/TLT.2023.3326473
K. Leiden, K. R. Laughery, J. Keller, J. French, W.Warwick, and S. D. Wood. A review of human performance models for the prediction of human error. Ann Arbor, 1001:48105, 2001.
M. G. Lenné, K. Ashby, and M. Fitzharris. Analysis of General Aviation Crashes in Australia Using the Human Factors Analysis and Classification System. The International Journal of Aviation Psychology, 18(4):340-352, 2008. doi: 10.1080/10508410802346939
C.-J. Lin, C. Wu, and W. A. Chaovalitwongse. Integrating human behavior modeling and data mining techniques to predict human errors in numerical typing. IEEE Transactions on Human-Machine Systems, 45(1):39-50, 2014. doi: 10.1109/THMS.2014.2357178
M. A. Makary and M. Daniel. Medical error-the third leading cause of death in the us. BMJ, 353, 2016. doi: 10.1136/bmj.i2139
S. Niknam and J. Botev. Oopses and Ohs: An Eye-tracking Study of Human Errors in Virtual Reality, 2026. doi: 10.5281/zenodo.18376588
Z. Nivolianitou, M. Konstandinidou, and C. Michalis. Statistical analysis of major accidents in petrochemical industry notified to the major accident reporting system (mars). Journal of hazardous materials, 137(1):1-7, 2006. doi: 10.1016/j.jhazmat.2004.12.042
Y. J. Oh, Y. H. Lee, and J. H. Yun. A study on the operator's erroneous responses to the new human interface of a digital device to be introduced to nuclear power plants. In International Conference on Human-Computer Interaction, pp. 337-341. Springer, 2011. doi: 10. 1007/978-3-642-22098-2 68
S. Panesar, Y. Cagle, D. Chander, J. Morey, J. Fernandez-Miranda, and M. Kliot. Artificial intelligence and the future of surgical robotics. Annals of surgery, 270(2):223-226, 2019. doi: 10.1097/SLA. 0000000000003262
J. Rasmussen. Concepts and structures in models of human process operators, 1979.
J. Reason. Human Error. Cambridge university press, 1990. doi: 10. 1017/cbo9781139062367 1
G. Ross and A. Gilbey. Extended reality (xr) flight simulators as an adjunct to traditional flight training methods: A scoping review. CEAS Aeronautical Journal, 14(4):799-815, 2023. doi: 10.1007/s13272-023-00688-5
R. Seil, C. Hoeltgen, H. Thomazeau, H. Anetzberger, and R. Becker. Surgical simulation training should become a mandatory part of orthopaedic education. Journal of Experimental Orthopaedics, 9(1):22, 2022. doi: 10.1186/s40634-022-00455-1
G. J. Siegle, N. Ichikawa, and S. Steinhauer. Blink before and after you think: Blinks occur prior to and following cognitive load indexed by pupillary responses. Psychophysiology, 45(5):679-687, 2008. doi: 10.1111/j.1469-8986.2008.00681.x
N. A. Stanton and P. M. Salmon. Human error taxonomies applied to driving: A generic driver error taxonomy and its implications for intelligent transport systems. Safety Science, 47(2):227-237, 2009. doi: 10.1016/j.ssci.2008.03.006
N. Takada, T. Laohakangvalvit, and M. Sugaya. Human error prediction using heart rate variability and electroencephalography. Sensors, 22(23):9194, 2022. doi: 10.3390/s22239194
A. E. Urai, A. Braun, and T. H. Donner. Pupil-linked arousal is driven by decision uncertainty and alters serial choice bias. Nature communications, 8(1):14637, 2017. doi: 10.1038/ncomms146373
M. E. Van Bochove, L. Van Der Haegen, W. Notebaert, and T. Verguts. Blinking predicts enhanced cognitive control. Cognitive, Affective, & Behavioral Neuroscience, 13(2):346-354, 2013. doi: 10.3758/s13415-012-0138-2
J. N. K. Wah. Revolutionizing surgery: Ai and robotics for precision, risk reduction, and innovation. Journal of Robotic Surgery, 19(1):47, 2025. doi: 0.1007/s11701-024-02205-0
T. Wendler, F. W. B. Van Leeuwen, N. Navab, and M. N. Van Oosterom. How molecular imaging will enable robotic precision surgery: The role of artificial intelligence, augmented reality, and navigation. European Journal of Nuclear Medicine and Molecular Imaging, 48(13):4201-4224, 2021. doi: 10.1007/s00259-021-05445-6
C.Wirth, P. M. Dockree, S. Harty, E. Lacey, and M. Arvaneh. Towards error categorisation in bci: Single-trial eeg classification between different errors. Journal of neural engineering, 17(1):016008, 2019. doi: 10.1088/1741-2552/ab53fe
M. Zhang, L. Dai, W. Chen, and E. Pang. Analysis of human errors in nuclear power plant event reports. Nuclear Engineering and Technology, 57(10):103687, 2025. doi: 10.1016/j.net.2025.103687.