[en] Preventing cheating without invading test-takers’ privacy in high-stakes online summative assessments poses a challenge, especially when the assessment is remote and unproctored. We conducted a between-subjects experiment (N = 997) in a realistic online test simulation to investigate the effects of three privacy-non-invasive anti-cheating interventions (honor code reminder, warning message, and monitoring message) on cheating prevention from a user-centered perspective. The quantitative results indicated that, compared to a control condition, displaying a honor code reminder during an online test worked best in lowering the odds of cheating. None of the interventions affected user experience and test-taking self-efficacy significantly. Further open-ended questions revealed that interventions can cause distraction which in turn could potentially evoke negative emotions. The decision to cheat was influenced by the extent to which interventions conveyed that cheating is wrong and also by test-takers’ perception of getting caught if they cheated. We derived recommendations for a fair and cheating-preventive unproctored online assessment for researchers and practitioners.
Disciplines :
Education & instruction
Author, co-author :
MUKHERJEE, Suvadeep ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > IRiSC
Rohles, Björn ; Human-Computer Interaction Research Group, University of Luxembourg, Esch-sur-Alzette, Luxembourg
DISTLER, Verena ; University of Luxembourg > Faculty of Humanities, Education and Social Sciences > Department of Behavioural and Cognitive Sciences > Team Vincent KOENIG
LENZINI, Gabriele ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > IRiSC
KOENIG, Vincent ; University of Luxembourg > Faculty of Humanities, Education and Social Sciences > Department of Behavioural and Cognitive Sciences > Team Vincent KOENIG
External co-authors :
yes
Language :
English
Title :
The effects of privacy-non-invasive interventions on cheating prevention and user experience in unproctored online assessments: An empirical study
FNR14926102 - Secure And Verifiable Electronic Testing And Assessment Systems, 2020 (01/05/2021-30/04/2025) - Gabriele Lenzini
Funders :
Agence Nationale de la Recherche Fonds National de la Recherche Luxembourg
Funding text :
This research is the result of the project “Secure and Verifiable Electronic Testing and Assessment Systems” (INTER/ANR/20/14926102/SEVERITAS) funded by the Luxembourg National Research Fund (FNR) and the French National Research Agency (ANR) .
Amigud, A., Lancaster, T., 246 reasons to cheat: An analysis of students' reasons for seeking to outsource academic work. Computers & Education 134 (2019), 98–107.
Balash, D.G., Kim, D., Shaibekova, D., Fainchtein, R.A., Sherr, M., Aviv, A.J., Examining the examiners: Students' privacy and security perceptions of online proctoring services. Seventeenth symposium on useable privacy and security, 2021, SOUPS 2021, 633–652.
Bandura, A., Social foundations of thought and action: A social cognitive theory. 1986, Princeton hall, englewood cliffs, nj.
Bandura, A., Selective activation and disengagement of moral control. Journal of Social Issues 46:1 (1990), 27–46.
Bandura, A., Perceived self-efficacy in cognitive development and functioning. Educational Psychologist 28:2 (1993), 117–148.
Bandura, A., Freeman, W.H., Lightsey, R., Self-efficacy: The exercise of control. 1999.
Barnhardt, B., The “epidemic” of cheating depends on its definition: A critique of inferring the moral quality of “cheating in any form”. Ethics & Behavior 26:4 (2016), 330–343.
Baume, M., Online proctored exams: Where and how are they used. Basics, practical scenarios and technical solutions for online proctoring at european universities and educational institutions, 2019, 13th International Technology, Education and Development Conference, 5216–5225.
Behrend, T.S., Sharek, D.J., Meade, A.W., Wiebe, E.N., The viability of crowdsourcing for survey research. Behavior Research Methods 43:3 (2011), 800–813.
Bing, M.N., Davison, H.K., Vitell, S.J., Ammeter, A.P., Garner, B.L., Novicevic, M.M., An experimental investigation of an interactive model of academic cheating among business school students. The Academy of Management Learning and Education 11:1 (2012), 28–48.
Brimble, M., Why students cheat: An exploration of the motivators of student academic dishonesty in higher education. 2016, Handbook of academic integrity.
Brinthaupt, T.M., Providing a realistic course preview to students. Teaching of Psychology 31:2 (2004), 104–106.
Buck, R., Davis, W.A., Marketing risk: Emotional appeals can promote the mindless acceptance of risk. Emotions and risky technologies, 2010, Springer, 61–80.
Buck, R., Ferrer, R., Emotion, warnings, and the ethics of risk communication. Handbook of risk theory, 2012, 694–723.
Butler-Henderson, K., Crawford, J., A systematic review of online examinations: A pedagogical innovation for scalable authentication and integrity. Computers & Education, 159, 2020, 104024.
Chandler, J., Rosenzweig, C., Moss, A.J., Robinson, J., Litman, L., Online panels in social science research: Expanding sampling methods beyond mechanical turk. Behavior Research Methods 51:5 (2019), 2022–2038.
Chen, G., Gully, S.M., Eden, D., Validation of a new general self-efficacy scale. Organizational Research Methods 4:1 (2001), 62–83.
Chen, B., West, M., Zilles, C., How much randomization is needed to deter collaborative cheating on asynchronous exams?. Proceedings of the fifth annual ACM conference on learning at scale, 2018, 1–10.
Chin, E.C., Williams, M.W., Taylor, J.E., Harvey, S.T., The influence of negative affect on test anxiety and academic performance: An examination of the tripartite model of emotions. Learning and Individual Differences 54 (2017), 1–8.
Cohney, S., Teixeira, R., Kohlbrenner, A., Narayanan, A., Kshirsagar, M., Shvartzshnaider, Y., Sanfilippo, M., Virtual classrooms and real harms: Remote learning at {US}. universities. Seventeenth symposium on useable privacy and security, 2021, SOUPS 2021), 653–674.
Conijn, R., Kleingeld, A., Matzat, U., Snijders, C., The fear of big brother: The potential negative side-effects of proctored exams. Journal of Computer Assisted Learning 38:6 (2022), 1521–1534.
Coohey, C., Cummings, S.P., Evaluation of an online group intervention to improve test-taking self-efficacy and reduce licensure test anxiety. Journal of Social Work Education 55:2 (2019), 376–388.
Corrigan-Gibbs, H., Gupta, N., Northcutt, C., Cutrell, E., Thies, W., Deterring cheating in online environments. ACM Transactions on Computer-Human Interaction 22:6 (2015), 1–23.
Credé, M., Phillips, L.A., A meta-analytic review of the motivated strategies for learning questionnaire. Learning and Individual Differences 21:4 (2011), 337–346.
Creswell, J.W., Poth, C.N., Qualitative inquiry and research design: Choosing among five approaches. 2016, Sage publications.
Daffin, L.W. Jr., Jones, A.A., Comparing student performance on proctored and non-proctored exams in online psychology courses. Online Learning 22:1 (2018), 131–145.
Dawson, P., Sutherland-Smith, W., Can training improve marker accuracy at detecting contract cheating? A multi-disciplinary pre-post study. Assessment & Evaluation in Higher Education 44:5 (2019), 715–725.
Doherty, S., Nelson, R., Using projective techniques to tap into consumers' feelings, perceptions and attitudes. getting an honest opinion. International Journal of Consumer Studies 34:4 (2010), 400–404.
Donoghue, S., Projective techniques in consumer research. Journal of Consumer Sciences, 28, 2000.
Eden, D., Pygmalion, goal setting, and expectancy: Compatible ways to boost productivity. Academy of Management Review 13:4 (1988), 639–652.
Eden, D., From self-efficacy to means efficacy: Internal and external sources of general and specific efficacy. 56th annual meeting of the Academy of Management, 1996 Cincinnati, OH.
Eden, D., Zuk, Y., Seasickness as a self-fulfilling prophecy: Raising self-efficacy to boost performance at sea. Journal of Applied Psychology, 80(5), 1995, 628.
Fazio, R.H., Cooper, J., Arousal in the dissonance process. Social psychophysiology: A sourcebook, 1983, 122–152.
Fontaine, S., Frenette, E., Hébert, M.-H., Exam cheating among quebec's preservice teachers: The influencing factors. International Journal for Educational Integrity 16:1 (2020), 1–18.
Freiburger, T.L., Romain, D.M., Randol, B.M., Marcum, C.D., Cheating behaviors among undergraduate college students: Results from a factorial survey. Journal of Criminal Justice Education 28:2 (2017), 222–247.
Ghanem, C.M., Mozahem, N.A., A study of cheating beliefs, engagement, and perception–the case of business and engineering students. Journal of Academic Ethics 17 (2019), 291–312.
Harper, R., Bretag, T., Ellis, C., Newton, P., Rozenberg, P., Saddiqui, S., van Haeringen, K., Contract cheating: A survey of australian university staff. Studies in Higher Education 44:11 (2019), 1857–1873.
Hassenzahl, M., The effect of perceived hedonic quality on product appealingness. International Journal of Human-Computer Interaction 13:4 (2001), 481–499.
Hassenzahl, M., Burmester, M., Koller, F., Attrakdiff: Ein fragebogen zur messung wahrgenommener hedonischer und pragmatischer qualität. Mensch & computer 2003, 2003, Springer, 187–196.
Hassenzahl, M., Tractinsky, N., User experience-a research agenda. Behaviour & Information Technology 25:2 (2006), 91–97.
Honicke, T., Broadbent, J., The influence of academic self-efficacy on academic performance: A systematic review. Educational Research Review 17 (2016), 63–84.
Hu, S., Jia, X., Fu, Y., Research on abnormal behavior detection of online examination based on image information. 2018 10th international conference on intelligent human-machine systems and cybernetics (IHMSC), Vol. 2, 2018, IEEE, 88–91.
Humbert, M., Lambin, X., Villard, E., The role of prior warnings when cheating is easy and punishment is credible. Information Economics and Policy, 58, 2022, 100959.
Hylton, K., Levy, Y., Dringus, L.P., Utilizing webcam-based proctoring to deter misconduct in online exams. Computers & Education 92 (2016), 53–63.
Jordan, A.E., College student cheating: The role of motivation, perceived norms, attitudes, and knowledge of institutional policy. Ethics & Behavior 11:3 (2001), 233–247.
Kam, C.C.S., Hue, M.T., Cheung, H.Y., Academic dishonesty among Hong Kong secondary school students: Application of theory of planned behaviour. Educational Psychology 38:7 (2018), 945–963.
Karim, M.N., Kaminsky, S.E., Behrend, T.S., Cheating, reactions, and performance in remotely proctored testing: An exploratory experimental study. Journal of Business and Psychology 29:4 (2014), 555–572.
Klass, E.T., Psychological effects of immoral actions: The experimental evidence. Psychological Bulletin, 85(4), 1978, 756.
Kujala, S., Walsh, T., Nurkka, P., Crisan, M., Sentence completion for understanding users and evaluating user experience. Interacting with Computers 26:3 (2014), 238–255.
Ladyshewsky, R.K., Post-graduate student performance in ‘supervised in-class’ vs.‘unsupervised online'multiple choice tests: Implications for cheating and test security. Assessment & Evaluation in Higher Education 40:7 (2015), 883–897.
Laugwitz, B., Held, T., Schrepp, M., Construction and evaluation of a user experience questionnaire. Symposium of the Austrian HCI and usability engineering group, 2008, Springer, 63–76.
Lerner, M.J., The justice motive: Some hypotheses as to its origins and forms 1. Journal of Personality 45:1 (1977), 1–52.
Lilley, M., Meere, J., Barker, T., Remote live invigilation: A pilot study. Journal of Interactive Media in Education, 2016(1), 2016.
Li, H., Xu, M., Wang, Y., Wei, H., Qu, H., A visual analytics approach to facilitate the proctoring of online exams. Proceedings of the 2021 CHI conference on human factors in computing systems, 2021, 1–17.
Lown, J.M., Development and validation of a financial self-efficacy scale. Journal of Financial Counseling and Planning, 22(2), 2011, 54.
Malesky, A., Grist, C., Poovey, K., Dennis, N., The effects of peer influence, honor codes, and personality traits on cheating behavior in a university setting. Ethics & Behavior 32:1 (2022), 12–21.
McCabe, D.L., Faculty responses to academic dishonesty: The influence of student honor codes. Research in Higher Education 34:5 (1993), 647–658.
McCabe, D.L., Trevino, L.K., Academic dishonesty: Honor codes and other contextual influences. The Journal of Higher Education 64:5 (1993), 522–538.
McCabe, D.L., Trevino, L.K., Butterfield, K.D., Dishonesty in academic environments: The influence of peer reporting requirements. The Journal of Higher Education 72:1 (2001), 29–45.
McDonald, N., Schoenebeck, S., Forte, A., Reliability and inter-rater reliability in qualitative research: Norms and guidelines for cscw and hci practice. Proceedings of the ACM on human-computer interaction 3:CSCW (2019), 1–23.
Miller, A., Shoptaugh, C., Wooldridge, J., Reasons not to cheat, academic-integrity responsibility, and frequency of cheating. The Journal of Experimental Education 79:2 (2011), 169–184.
Milone, A.S., Cortese, A.M., Balestrieri, R.L., Pittenger, A.L., The impact of proctored online exams on the educational experience. Currents in Pharmacy Teaching and Learning 9:1 (2017), 108–114.
Minge, M., Riedel, L., mecue-ein modularer fragebogen zur erfassung des nutzungserlebens. Mensch & computer, 2013, 89–98.
Mitchell, M.S., Baer, M.D., Ambrose, M.L., Folger, R., Palmer, N.F., Cheating under pressure: A self-protection model of workplace cheating behavior. Journal of Applied Psychology, 103(1), 2018, 54.
Mlekus, L., Bentler, D., Paruzel, A., Kato-Beiderwieden, A.-L., Maier, G.W., How to raise technology acceptance: User experience characteristics as technology-inherent determinants. Gruppe. Interaktion. Organisation. Zeitschrift für Angewandte Organisationspsychologie (GIO) 51:3 (2020), 273–283.
Moten, J. Jr., Fitterer, A., Brazier, E., Leonard, J., Brown, A., Examining online college cyber cheating methods and prevention measures. Electronic Journal of e-Learning 11:2 (2013), pp139–146.
Nagin, D.S., Pogarsky, G., An experimental investigation of deterrence: Cheating, self-serving bias, and impulsivity. Criminology 41:1 (2003), 167–194.
Park, S., Avery, E.J., Development and validation of a crisis self-efficacy index. Journal of Contingencies and Crisis Management 27:3 (2019), 247–256.
Peer, E., Rothschild, D., Gordon, A., Evernden, Z., Damer, E., Data quality of platforms and panels for online behavioral research. Behavior Research Methods 54:4 (2022), 1643–1662.
Pleasants, J., Pleasants, J.M., Pleasants, B.P., Cheating on unproctored online exams: Prevalence, mitigation measures, and effects on exam performance. Online Learning, 26(1), 2022.
Ranger, J., Schmidt, N., Wolgast, A., The detection of cheating on e-exams in higher education—the performance of several old and some new indicators. Frontiers in Psychology, 11, 2020, 568825.
Rios, J.A., Liu, O.L., Online proctored versus unproctored low-stakes internet test administration: Is there differential test-taking behavior and performance?. American Journal of Distance Education 31:4 (2017), 226–241.
Schrepp, D.M., User experience questionnaire handbook. 8 edition, 2019.
Schrepp, M., Hinderks, A., Thomaschewski, J., Design and evaluation of a short version of the user experience questionnaire (ueq-s). International Journal of Interactive Multimedia and Artificial Intelligence 4:6 (2017), 103–108.
Schultz, M., Lim, K.F., Goh, Y.K., Callahan, D.L., Ok google: what's the answer? Characteristics of students who searched the internet during an online chemistry examination. 2022, Assessment & Evaluation in Higher Education.
Schultz, M., Schultz, J., Round, G., Online non-proctored testing and its affect on final course grades, Vol. 9, 2008, Business Rev, Cambridge, 11–16.
Schwartz, B.M., Tatum, H.E., Hageman, M.C., College students' perceptions of and responses to cheating at traditional, modified, and non-honor system institutions. Ethics & Behavior 23:6 (2013), 463–476.
Schwarzer, R., Jerusalem, M., Generalized self-efficacy scale. Weinman, J., Wright, S., Johnston, M., (eds.) Measures in health psychology: A user's portfolio. Causal and control beliefs, Vol. 35, 1995, 37.
Schwarzer, R., Jerusalem, M., The general self-efficacy scale (gse). Anxiety, Stress & Coping 12:1 (2010), 329–345.
Sefcik, L., Veeran-Colton, T., Baird, M., Price, C., Steyn, S., An examination of student user experience (ux) and perceptions of remote invigilation during online assessment. Australasian Journal of Educational Technology 38:2 (2022), 49–69.
Shaffer, D.R., Some effects of consonant and dissonant attitudinal advocacy on initial attitude salience and attitude change. Journal of Personality and Social Psychology, 32(1), 1975, 160.
Sherer, M., Maddux, J.E., Mercandante, B., Prentice-Dunn, S., Jacobs, B., Rogers, R.W., The self-efficacy scale: Construction and validation. Psychological Reports 51:2 (1982), 663–671.
Shu, L.L., Gino, F., Sweeping dishonesty under the rug: How unethical actions lead to forgetting of moral rules. Journal of Personality and Social Psychology, 102(6), 2012, 1164.
Stanley, K.D., Murphy, M.R., A comparison of general self-efficacy with self-esteem. Genetic, Social, and General Psychology Monographs 123:1 (1997), 79–100.
Thacker, E.J., et al. PhD thesis. Contract cheating and academic literacies: Exploring the landscape, 2022, Keele University.
Theeuwes, J., Perceptual selectivity for color and form. Perception & Psychophysics 51:6 (1992), 599–606.
Theeuwes, J., Kramer, A.F., Hahn, S., Irwin, D.E., Our eyes do not always go where we want them to go: Capture of the eyes by new objects. Psychological Science 9:5 (1998), 379–385.
Truxillo, D.M., Bauer, T.N., Sanchez, R.J., Multiple dimensions of procedural justice: Longitudinal effects on selection system fairness and test-taking self-efficacy. International Journal of Selection and Assessment 9:4 (2001), 336–349.
Watson, D., Clark, L.A., Tellegen, A., Development and validation of brief measures of positive and negative affect: The panas scales. Journal of Personality and Social Psychology, 54(6), 1988, 1063.
Wuthisatian, R., Student exam performance in different proctored environments: Evidence from an online economics course. International Review of Economics Education, 35, 2020, 100196.
Yucas, A., Chinese nationals charged with cheating by impersonation on us college tests. 2015, The Guardian.
Zeidner, M., Test anxiety: The state of the art. 1998.
Zimmerman, B.J., Self-efficacy and educational development. Self-efficacy in changing societies 1:1 (1995), 202–231.
