[en] The use of the Experience Sampling Method (ESM), which involves repeated assessments in people's daily lives, has increased in popularity in psychology and associated disciplines in recent years. A rather challenging aspect of ESM is its technical implementation. In this paper, after briefly introducing the history of ESM and the main reasons for its current popularity, we outline the ESM-Quest experience sampling app which is currently being developed at the University of Vienna. ESM-Quest runs on different operating systems, specifically on mobile devices such as smartphones and tablets running either iOS or Android. An internet connection is not necessary during the assessment. Compared to most other ESM apps, ESM-Quest allows event-based random sampling, which is very helpful when assessments need to be collected within specific situations. Currently, ESM-Quest is being utilized at the University of Vienna and will be made available for research groups worldwide upon request. We introduce the technical aspects of ESM-Quest and provide examples of analyses on ESM data collected through this app, such as examining fluctuations in constructs within individuals. Finally, we outline potential next steps in ESM research.
Disciplines :
Education & instruction
Author, co-author :
Goetz, Thomas; Department of Developmental and Educational Psychology, Faculty of Psychology, University of Vienna, Vienna, Austria
Steiner, Wolfgang; Department of Developmental and Educational Psychology, Faculty of Psychology, University of Vienna, Vienna, Austria
Graf, Elisabeth; Department of Developmental and Educational Psychology, Faculty of Psychology, University of Vienna, Vienna, Austria
Stempfer, Lisa; Department of Developmental and Educational Psychology, Faculty of Psychology, University of Vienna, Vienna, Austria
Ristl, Christina; Department of Developmental and Educational Psychology, Faculty of Psychology, University of Vienna, Vienna, Austria
Rupprecht, Fiona S; Department of Developmental and Educational Psychology, Faculty of Psychology, University of Vienna, Vienna, Austria
Donath, Johanna L; Department of Developmental and Educational Psychology, Faculty of Psychology, University of Vienna, Vienna, Austria
BOTES, Elouise ; University of Luxembourg > Faculty of Humanities, Education and Social Sciences (FHSE) > Department of Behavioural and Cognitive Sciences (DBCS) > Cognitive Science and Assessment ; Department of Developmental and Educational Psychology, Faculty of Psychology, University of Vienna, Vienna, Austria
Nikitin, Jana; Department of Developmental and Educational Psychology, Faculty of Psychology, University of Vienna, Vienna, Austria
External co-authors :
yes
Language :
English
Title :
Assessing psychological variables on mobile devices: an introduction to the experience sampling app ESM-Quest.
Asseburg R. Frey A. (2013). Too hard, too easy, or just right? The relationship between effort or boredom and ability-difficulty fit. Psychol. Test Assess. Model. 55, 92–104.
Beal D. J. (2015). ESM 2.0: state of the art and future potential of experience sampling methods in organizational research. Annu. Rev. Organ. Psych. Organ. Behav. 2, 383–407. doi: 10.1146/annurev-orgpsych-032414-111335
Bevans G. E. (1913). How workingmen spend their time. New York: Columbia University Press.
Conner T. S. Barrett L. F. (2012). Trends in ambulatory self-report: the role of momentary experience in psychosomatic medicine. Psychosom. Med. 74, 327–337. doi: 10.1097/PSY.0b013e3182546f18, PMID: 22582330
Consolvo S. Walker M. (2003). Using the experience sampling method to evaluate ubicomp applications. IEEE Perv. Comp. 2, 24–31. doi: 10.1109/MPRV.2003.1203750
Csikszentmihalyi M. Larson R. (1987). Validity and reliability of the experience sampling method. J. Nerv. Ment. Dis. 175, 526–536. doi: 10.1097/00005053-198709000-00004
Doherty K. Balaskas A. Doherty G. (2020). The design of ecological momentary assessment technologies. Interact. Comput. 32, 257–278. doi: 10.1093/iwcomp/iwaa019
Ebner-Priemer U. W. Trull T. J. (2009). Ecological momentary assessment of mood disorders and mood dysregulation. Psychol. Assess. 21, 463–475. doi: 10.1037/a0017075
Gabriel A. S. Podsakoff N. P. Beal D. J. Scott B. A. Sonnentag S. Trougakos J. P. et al. (2019). Experience sampling methods: a discussion of critical trends and considerations for scholarly advancement. Organ. Res. Methods 22, 969–1006. https://10.1177/1094428118802626. doi: 10.1177/1094428118802626
Goetz T. Bieg M. Hall N. C. (2016a). “Assessing academic emotions via the experience sampling method” in Methodological advances in research on emotion in education. eds. Zembylas M. Schutz P. (London: Springer)
Goetz T. Bieg M. Lüdtke O. Pekrun R. Hall N. C. (2013). Do girls really experience more anxiety in mathematics? Psychol. Sci. 24, 2079–2087. doi: 10.1177/095679761348698, PMID: 23985576
Goetz T. Bieleke M. Yanagida T. Krannich M. Roos A.-L. Frenzel A. C. et al. (2023). Test boredom: exploring a neglected emotion. J. Educ. Psychol. 115, 911–931. doi: 10.1037/edu0000807
Goetz T. Frenzel A. C. Pekrun R. Hall N. C. Lüdtke O. (2007). Between- and within-domain relations of students’ academic emotions. J. Educ. Psychol. 99, 715–733. doi: 10.1037/0022-0663.99.4.715
Goetz T. Frenzel A. C. Stockinger K. Lipnevich A. A. Stempfer L. Pekrun R. (2023). “Emotions in education” in International encyclopedia of education. eds. Tierney R. J. Rizvi F. Erkican K.. 4th ed (Amsterdam: Elsevier)
Goetz T. Sticca F. Pekrun R. Murayama K. Elliot A. J. (2016b). Intraindividual relations between achievement goals and discrete achievement emotions: an experience sampling approach. Learn. Instr. 41, 115–125. doi: 10.1016/j.learninstruc.2015.10.007
Gogol K. Brunner M. Goetz T. Martin R. Ugen S. Fischbach A. et al. (2014). ‘My questionnaire is too long!’ The assessments of motivational-affective constructs with three-item and single-item measures. Contemp. Educ. Psychol. 39, 188–205. doi: 10.1016/j.cedpsych.2014.04.002
Haedt-Matt A. A. Keel P. K. (2011). Revisiting the affect regulation model of binge eating: a meta-analysis of studies using ecological momentary assessment. Psychol. Bull. 137, 660–681. doi: 10.1037/a0023660, PMID: 21574678
Hamilton J. D. (2020). Time series analysis. Princeton: Princeton University Press.
Harley J. M. Bouchet F. Hussain M. S. Azevedo R. Calvo R. (2015). A multi-componential analysis of emotions during complex learning with an intelligent multi-agent system. Comput. Hum. Behav. 48, 615–625. doi: 10.1016/j.chb.2015.02.013
Lamiell J. T. (1997). “Individuals and the differences between them” in Handbook of personality psychology. eds. Hogan R. Johnson J. A. Briggs S. R. (Cambridge: Academic Press)
Mac Donald S. W. S. Stawski R. S. (2014). “Intraindividual variability—an indicator of vulnerability or resilience in adult development and aging?” in Handbook of Intraindividual variability across the life span. eds. Diehl M. Hooker K. Sliwinski M. J. (London: Routledge)
Mitchell T. R. Thompson L. Peterson E. Cronk R. (1997). Temporal adjustments in the evaluation of events: the “rosy view”. J. Exp. Soc. Psychol. 33, 421–448. doi: 10.1006/jesp.1997.1333
Murayama K. Goetz T. Malmberg L.-E. Pekrun R. Tanaka A. Martin A. J. (2017). “Within-person analysis in educational psychology: importance and illustrations” in British Journal of Educational Psychology monograph series II: Psychological aspects of education - current trends: the role of competence beliefs in teaching and learning. eds. Putwain D. W. Smart K. (Hoboken: Wiley)
Pekrun R. Marsh H. W. Elliot A. J. Stockinger K. Perry R. P. Vogl E. et al. (2023). A three-dimensional taxonomy of achievement emotions. J. Pers. Soc. Psychol. 124, 145–178. doi: 10.1037/pspp0000448, PMID: 36521161
Rauthmann J. F. Horstmann K. T. Sherman R. A. (2019). Do self-reported traits and aggregated states capture the same thing? A Nomological perspective on trait-state homomorphy. Soc. Psychol. Personal. Sci. 10, 596–611. doi: 10.1177/1948550618774772
Röcke C. Li S. C. Smith J. (2009). Intraindividual variability in positive and negative affect over 45 days: do older adults fluctuate less than young adults? Psychol. Aging 24, 863–878. doi: 10.1037/a0016276
Roos A. Goetz T. Krannich M. Donker M. Bieleke M. Caltabiano A. et al. (2023). Control, anxiety and test performance: self-reported and physiological indicators of anxiety as mediators. Br. J. Educ. Psychol. 93, 72–89. doi: 10.1111/bjep.12536, PMID: 35906734
Scollon C. N. Kim-Prieto C. Diener E. (2003). Experience sampling: promises and pitfalls, strengths and weaknesses. J. Happiness Stud. 4, 5–34. doi: 10.1023/A:1023605205115
Shiffman S. (2009). Ecological momentary assessment (EMA) in studies of substance use. Psychol. Assess. 21, 486–497. doi: 10.1037/a0017074, PMID: 19947783
Silvia P. J. Kwapil T. R. Eddington K. M. Brown L. H. (2013). Missed beeps and missing data: dispositional and situational predictors of nonresponse in experience sampling research. Soc. Sci. Comput. Rev. 31, 471–481. doi: 10.1177/0894439313479902
Sliwinski M. J. (2008). Measurement-burst designs for social health research. Soc. Personal. Psychol. Compass 2, 245–261. doi: 10.1111/j.1751-9004.2007.00043.x
Trull T. J. Ebner-Priemer U. (2014). The role of ambulatory assessment in psychological science. Curr. Dir. Psychol. Sci. 23, 466–470. doi: 10.1177/0963721414550706, PMID: 25530686
Van Berkel N. Ferreira D. Kostakos V. (2017). The experience sampling method on mobile devices. ACM Comput. Surv. 50, 1–40. doi: 10.1145/3123988
Voelkle M. C. Brose A. Schmiedek F. Lindenberger U. (2014). Towards a unified framework for the study of between-person and within-person structures: building a bridge between two research paradigms. Multivar. Behav. Res. 49, 193–213. doi: 10.1080/00273171.2014.889593, PMID: 26735189
Wainer H. (2000). Computerized adaptive testing: A primer (2nd). Mahwah: Lawrence Erlbaum Associates.
Wrzus C. Neubauer A. B. (2023). Ecological momentary assessment: a meta-analysis on designs, samples, and compliance across research fields. Assessment 30, 825–846. doi: 10.1177/10731911211067538, PMID: 35016567
