Contribution à des ouvrages collectifs (Parties d’ouvrages)
Is There More Than the Answer to the Question? Device Use and Completion Time as Indicators for Selectivity Bias and Response Convenience in Online Surveys
DECIEUX, Jean Philippe Pierre
2021 • In Erlinghagen, Marcel; Ette, Andreas; Schneider, Norbertet al. (Eds.) The Global Lives of German Migrants
[en] The main objective of the German Emigration and Remigration Panel Study (GERPS) is to establish a longitudinal data set that provides information on life trajectories of international migrants. However, a large amount of paradata were also collected in order to obtain meta-information on respondents’ survey participation. This auxiliary information can help to optimize data quality at all stages of the survey process. By continuing the existing discussion in the field of online surveys, this chapter pursues a twofold objective: it reflects device usage (mobile vs. computer) and elucidates determinants of device choice. In particular, it analyses whether selectivity effects due to respondent’s device choices bias the sample. Moreover, this chapter investigates differences in response time between devices to detect differences in response burden. The analysis of response burden differences by device is an important issue, since an increased device-specific response burden can be a predictor of actual and further panel dropouts. In both device-specific selectivity and survey burden, only slight differences were found between mobile and desktop devices. Using these data, the following paper addresses the need to analyse potential sources of survey error and provides evidence that GERPS data do not appear to contain noteworthy bias attributed to device usage.
Disciplines :
Sociologie & sciences sociales
Auteur, co-auteur :
DECIEUX, Jean Philippe Pierre ; University of Luxembourg > Faculty of Language and Literature, Humanities, Arts and Education (FLSHASE) > Integrative Research Unit: Social and Individual Development (INSIDE)
Co-auteurs externes :
yes
Langue du document :
Anglais
Titre :
Is There More Than the Answer to the Question? Device Use and Completion Time as Indicators for Selectivity Bias and Response Convenience in Online Surveys
Andreadis, I. (2015). Web surveys optimized for smartphones: Are there differences between computer and smartphone users? Methods, Data, Analyses, 9(2), 213–228.
Antoun, C., & Cernat, A. (2019). Factors affecting completion times: A comparative analysis of smartphone and PC web surveys. Social Science Computer Review, 38, 477.
Antoun, C., Couper, M. P., & Conrad, F. G. (2017). Effects of mobile versus PC web on survey response quality: A crossover experiment in a probability web panel. Public Opinion Quarterly, 81(S1), 280–306.
Brockhaus, S., Keusch, F., Henninger, F., Horwitz, R., Kieslich, P., Kreuter, F., and Schierholz, M. (2017). Learning from mouse movements: Improving web questionnaire and respondents' user experience through passive data collection. Miami.
Cook, W. A. (2014). Is mobile a reliable platform for survey taking? Defining quality in online surveys from mobile respondents. Journal of Advertising Research, 54(2), 131–148.
Couper, M. P. (2008). Designing effective web surveys. Cambridge: Cambridge University Press.
Couper, M. P., Antoun, C., & Mavletova, A. (2017). Mobile web surveys: A total survey error perspective. In P. P. Biemer, E. De Leeuw, S. Eckman, B. Edwards, F. Kreuter, L. E. Lyberg, C. Tucker, & B. T. West (Eds.), Total survey error in practice (pp. 133–154). Hoboken, NJ: John Wiley & Sons, Inc.
Couper, M. P., & Peterson, G. J. (2017). Why do web surveys take longer on smartphones? Social Science Computer Review, 35(3), 357–377.
de Bruijne, M., & Wijnant, A. (2013). Comparing survey results obtained via mobile devices and computers: An experiment with a mobile web survey on a heterogeneous group of mobile devices versus a computer-assisted web survey. Social Science Computer Review, 31(4), 482–504.
Decieux, J. P. P., Witte, N., Ette, A., Erlinghagen, M., Guedes Auditor, J., Sander, N., and Schneider, N. (2019). Individual Consequences of Migration in a Life Course Perspective: Experiences of the First Two Waves of the New German Emigration and Remigration Panel Study (GERPS). European Survey Research Association (ESRA) Conference, Zagreb, 2019.
Décieux, J. P., Heinen, A., & Willems, H. (2018). Social media and its role in friendship-driven interactions among young people: A mixed methods study. Young, 27(1), 18–31.
Décieux, J. P. P., & Hoffmann, M. (2014). Antwortdifferenzen im junk & crime survey: Ein Methodenvergleich mit goffmanscher Interpretation. In M. Löw (Ed.), Vielfalt und Zusammenhalt: Verhandlungen des 36. Kongresses der Deutschen Gesellschaft für Soziologie in Bochum und Dortmund 2012. Frankfurt am Main, New York: Campus.
Diedenhofen, B., & Musch, J. (2017). PageFocus: Using paradata to detect and prevent cheating on online achievement tests. Behavior Research Methods, 49, 1444–1459.
Durrant, G. B., D'Arrigo, J., & Müller, G. (2013). Modelling call recorded data: Examples from cross-sectional and longitudinal surveys. In F. Kreuter (Ed.), Improving surveys with paradata. Analytic uses of process information (pp. 281–308). Hoboken, NJ: John Wiley & Sons.
Erlinghagen, M., Ette, A., Schneider, N. F., Witte, N., & Décieux, J. P. (2019). Internationale migration zwischen hochentwickelten Staaten und ihre Konsequenzen für den Lebensverlauf. In N. Burzan (Ed.), Komplexe Dynamiken globaler und lokaler Entwicklungen. Essen: DGS.
Erzen, E., Odaci, H., & Yeniçeri, İ. (2019). Phubbing: Which personality traits are prone to phubbing? Social Science Computer Review.
Ette, A., & Erlinghagen, M. (2021). Structures of German Emigration and remigration: Historical developments and demographic patterns. In M. Erlinghagen, A. Ette, N. F. Schneider, & N. Witte (Eds.), Consequences of International Migration across the Life Course: Global Lives of German Migrants. Springer.
Ette, A., Décieux, J. P., Erlinghagen, M., Auditor, J. G., Sander, N., Schneider, N. F., & Witte, N. (2021). Surveying across borders: The experiences of the German emigration and remigration panel study. In M. Erlinghagen, A. Ette, N. F. Schneider, & N. Witte (Eds.), The global lives of German migrants. Consequences of international migration across the life course. Cham: Springer.
Ette, A., Décieux, J. P., Erlinghagen, M., Genoni, A., Auditor, J. G., Knirsch, F., Kühne, S., Mörchen, L., Sand, M., Schneider, N. F., & Witte, N. (2020). German emigration and remigration panel study (GERPS): Methodology and data manual of the baseline survey (wave 1). Wiesbaden: Bundesinstitut für Bevölkerungsforschung.
Ette, A., & Sauer, L. (2010). Auswanderung aus Deutschland. In Daten und Analysen zur internationalen Migration deutscher Staatsbürger. Wiesbaden: VS Verlag.
Gibson, A. M., & Bowling, N. A. (2020). The effects of questionnaire length and behavioral consequences on careless responding. European Journal of Psychological Assessment, 36, 410–420.
Groves, R. M., Singer, E., & Corning, A. (2000). Leverage-saliency theory of survey participation: Description and an illustration. Public Opinion Quarterly, 64(3), 299–308.
Gummer, T., Quoß, F., & Roßmann, J. (2019). Does increasing mobile device coverage reduce heterogeneity in completing web surveys on smartphones? Social Science Computer Review, 37(3), 371–384.
Gummer, T., & Roßmann, J. (2015). Explaining interview duration in web surveys: A multilevel approach. Social Science Computer Review, 33(2), 217–234.
Haan, M., Lugtig, P., & Toepoel, V. (2019). Can we predict device use? An investigation into mobile device use in surveys. International Journal of Social Research Methodology, 22(5), 517–531.
Herzing, J. M. E. (2019). Mobile web surveys. In F. O. R. S. Guide (Ed.), Swiss Centre of Expertise in the Social Sciences FORS. Lausanne: FORS.
Höhne, J. K., & Schlosser, S. (2018). Investigating the adequacy of response time outlier definitions in computer-based web surveys using paradata SurveyFocus. Social Science Computer Review, 36(3), 369–378.
Jacob, R., Heinz, A., & Décieux, J. P. (2019). Umfrage: Einführung in die Methoden der Umfrageforschung. Walter de Gruyter: Oldenburg.
Kreuter, F. (Ed.). (2013). Improving surveys with paradata. Analytic uses of process information. Hoboken, NJ: John Wiley & Sons.
Lambert, A. D., & Miller, A. L. (2015). Living with smartphones: Does completion device affect survey responses? Research in Higher Education, 56(2), 166–177.
Lee, H., Kim, S., Couper, M. P., & Woo, Y. (2018). Experimental comparison of PC web, smartphone web, and telephone surveys in the new technology era. Social Science Computer Review, 37(2), 234–247.
Leiner, D. J. (2019). Too fast, too straight, too weird: Non-reactive indicators for meaningless data in internet surveys. Survey Research Methods, 13(3), 229–248.
Mancosu, M., Ladini, R., & Vezzoni, C. (2019). 'Short is better'. Evaluating the attentiveness of online respondents through screener questions in a real survey environment. Bulletin of Sociological Methodology/Bulletin de Méthodologie Sociologique, 141(1), 30–45.
Mayerl, J., & Giehl, C. (2018). A closer look at attitude scales with positive and negative items. Response latency perspectives on measurement quality. Survey Research Methods, 12(3), 193–209.
McClain, C. A., Couper, M. P., Hupp, A. L., Keusch, F., Peterson, G. J., Piskorowski, A. D., & West, B. T. (2019). A typology of web survey paradata for assessing total survey error. Social Science Computer Review, 37(2), 196–213.
Mergener, A., & Décieux, J. P. P. (2018). Die “Kunst” des Fragenstellens. In B. Keller, H.-W. Klein, & T. Wirth (Eds.), Qualität und Data Science in der Marktforschung: Prozesse, Daten und Modelle der Zukunft (pp. 81–97). Wiesbaden: Springer Fachmedien Wiesbaden.
Montgomery, J. M., & Rossiter, E. L. (2020). So many questions, so little time: Integrating adaptive inventories into public opinion research. Journal of Survey Statistics and Methodology, 8(4), 667–690.
Peytchev, A. (2009). Survey breakoff. The Public Opinion Quarterly, 73(1), 74–97.
Revilla, M., Toninelli, D., Ochoa, C., & Loewe, G. (2016). Do online access panels need to adapt surveys for mobile devices? Internet Research, 26(5), 1209–1227.
Roßmann, J. (2015). RSPEEDINDEX: Stata module to compute a response speed index and perform outlier identification.
Rolstad, S., Adler, J., & Rydén, A. (2011). Response burden and questionnaire length: Is shorter better? A review and meta-analysis. Value in Health, 14(8), 1101–1108.
Roßmann, J., & Gummer, T. (2016). Using paradata to predict and correct for panel attrition. Social Science Computer Review, 34(3), 312–332.
Roßmann, J., Gummer, T., & Silber, H. (2018). Mitigating satisficing in cognitively demanding grid questions: Evidence from two web-based experiments. Journal of Survey Statistics and Methodology, 6(3), 376–400.
Schlosser, S., & Mays, A. (2018). Mobile and dirty: Does using mobile devices affect the data quality and the response process of online surveys? Social Science Computer Review, 36(2), 212–230.
Schnell, R., Hill, P. B., & Esser, E. (2013). Methoden der empirischen Sozialforschung (10th ed.). München: Oldenbourg.
Sendelbah, A., Vehovar, V., Slavec, A., & Petrovčič, A. (2016). Investigating repsondent multitasking in web surveys using paradata. Computers in Human Behavior, 55, 777–787.
Stieger, S., & Reips, U.-D. (2010). What are participants doing while filling in an online questionnaire: a paradata collection tool and an empirical study. Computers in Human Behavior, 26(6), 1488–1495.
Struminskaya, B., Weyandt, K., & Bosnjak, M. (2015). The effects of questionnaire completion using mobile devices on data quality. Evidence from a probability-based general population panel. Methods, Data, Analyses, 9(2), 261–292.
Thulin, E. (2018). Always on my mind: How smartphones are transforming social contact among young swedes. Young, 26(5), 465–483.
Tourangeau, R., Rips, L. J., & Rasinski, K. (2000). The psychology of survey response. Cambridge: Cambridge University Press.
Turkle, S. (2017). Alone together: Why we expect more from technology and less from each other. New York: Basic Books.
Verbree, A.-R., Toepoel, V., & Perada, D. (2019). The effect of seriousness and device use on data quality. Social Science Computer Review, 38, 720.
Villar, A., Callegaro, M., & Yang, Y. (2013). Where am I? A meta-analysis of experiments on the effects of progress indicators for web surveys. Social Science Computer Review, 31(6), 744–762.
Ward, M. K., & Meade, A. W. (2018). Applying social psychology to prevent careless responding during online surveys. Applied Psychology, 67(2), 231–263.
Wenz, A., Jäckle, A., & Couper, M. P. (2019). Willingness to use mobile technologies for data collection in a probability household panel. Survey Research Methods, 12(1), 1–22.
Yan, T., & Olson, K. (2013). Analyzing paradata to investigate measurement error. In F. Kreuter (Ed.), Improving surveys with paradata. Analytic uses of process information (pp. 73–96). Hoboken, NJ: John Wiley & Sons.
Zwarun, L., & Hall, A. (2014). What's going on? Age, distraction, and multitasking during online survey taking. Computers in Human Behavior, 41, 236–244.
