Oopses and Ohs: An Eye-Tracking Study of Human Errors in Virtual Reality

Human error remains a major source of accidents and casualties across domains, from manufacturing to healthcare. Building on Rasmussen’s Skill–Rule–Knowledge (SRK) model and Reason’s taxonomy, we investigate whether eye-tracking data can classify and predict cognitive error types. We conducted an 18-month longitudinal study in virtual reality (VR), collecting over 60 hours of eye-tracking data from two participants performing mental arithmetic. Samples were labeled as Correct, Missed, or Wrong and used to train an LSTM neural network. The model reliably separated correct from missed responses using pupil diameter, gaze dynamics, and blinks. However, wrong responses were frequently confused with correct ones, reflecting the challenge of detecting skill-based slips accompanied by false confidence. Predicting human error is especially critical when conventional practice is costly or risky, such as high-precision surgery or military operations. Our findings highlight the potential of eye-tracking data to advance the understanding, prediction, and prevention of human error.