Optimizing the Use of the Sentence Completion Survey Technique in User Research – A Case Study on the Experience of E-Reading

Lallemand, Carine; Mercier, Emeline

doi:10.1145/3491102.3517718

Paper published in a book (Scientific congresses, symposiums and conference proceedings)

Optimizing the Use of the Sentence Completion Survey Technique in User Research – A Case Study on the Experience of E-Reading

Lallemand, Carine; Mercier, Emeline

2022 • In Proceeding of the CHI 2022 Conference on Human Factors in Computing Systems

Peer reviewed

Permalink
https://hdl.handle.net/10993/50535

DOI
10.1145/3491102.3517718

Files (1)Send to Details Statistics Bibliography Similar publications

Files

Full Text

3491102.3517718.pdf

Publisher postprint (1.05 MB)

Download

All documents in ORBilu are protected by a user license.

Send to

RIS BibTex APA Chicago Permalink X Linkedin

Details

Abstract :

[en] Sentence completion, originally a semi-projective psychological technique, has been used as an effective and lightweight user research method in user experience (UX) design. More information is yet still needed to understand how different sentence stems probe users’ insights, thereby providing recommendations for effective sentence completion surveys. We used the completion method on a large-scale sample to explore (e-)readers’ experiences and needs. Depending on their reading habits, participants (N=1880) were asked to complete a set of sentences, as part of a web survey. With 14143 user ideas collected in two weeks, our results confirm that remote online sentence completion is a cost-effective data collection method able to uncover feelings, attitudes, motivations, needs, or frustrations. Variation in sentence stems affected collected data in terms of item response rate, idea quantity as well as variety and originality. Building on previous research, this paper delivers actionable insights to optimize the richness of sentence completion outputs.

Disciplines :

Social & behavioral sciences, psychology: Multidisciplinary, general & others

Author, co-author :

Lallemand, Carine ; University of Luxembourg > Faculty of Humanities, Education and Social Sciences (FHSE) > Department of Behavioural and Cognitive Sciences (DBCS)

Mercier, Emeline

External co-authors :

yes

Language :

English

Title :

Optimizing the Use of the Sentence Completion Survey Technique in User Research – A Case Study on the Experience of E-Reading

Publication date :

April 2022

Event name :

ACM CHI 2022 Conference on Human Factors in Computing Systems

Event date :

From 30-04-2022 to 05-05-2022

Audience :

International

Main work title :

Proceeding of the CHI 2022 Conference on Human Factors in Computing Systems

Publisher :

ACM

Peer reviewed :

Peer reviewed

Available on ORBilu :

since 10 March 2022

Statistics

Number of views

98 (6 by Unilu)

Number of downloads

22 (4 by Unilu)

More statistics

Scopus citations^®

Scopus citations^®
without self-citations

OpenCitations

Bibliography

Vero Vanden Abeele and Bieke Zaman. 2009. Laddering the User Experience! User Experience Evaluation Methods in Product Development (UXEM'09)-Workshop.
Margaret Ann. 1964. Use of the Gilmore Sentence Completion Test as a predictive instrument in relation to the academic achievement of certain high school students.
Kenneth S. Bordens and Bruce B. Abbott. 2011. Research Design and Methods. A process approach (8th edition). McGraw-Hill, New York, NY.
Kelly Caine. 2016. Local standards for sample size at CHI. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, Association for Computing Machinery New York, NY, USA, 981-992, DOI: 10.1145/2858036.2858498.
Carmen Camarero, Carmen Antón, and Javier Rodríguez. 2014. Technological and ethical antecedents of e-book piracy and price acceptance: Evidence from the Spanish case. The Electronic Library 32, 4: 542-566. DOI: 10.1108/EL-11-2012-0149
Nathan T. Carter, Michael A. Daniels, and Michael J. Zickar. 2013. Projective testing: Historical foundations and uses for human resources management. Human Resource Management Review 23, 3: 205-218. DOI: 10.1016/j.hrmr.2012.12.002
Tara Cataldo, Trey Shelton, Steven Carrico, and Cecilia Botero. 2014. How Users' Perceptions of E-Books Have Changed-Or Not: Comparing Parallel Survey Responses. Proceedings of the Charleston Library Conference, 304-310, DOI: 10.5703/1288284315587.
Gilbert Cockton, Sari Kujala, Piia Nurkka, and Taneli Hölttä. 2009. Supporting worth mapping with sentence completion. Lecture Notes in Computer Science (including subseries Lecture Notes in Artifcial Intelligence and Lecture Notes in Bioinformatics) 5727 LNCS, PART 2: 566-581. DOI: 10.1007/978-3-642-03658-3-61
Rue L. Cromwell and Richard M. Lundy. 1954. Productivity of Clinical Hypotheses on a Sentence Completion Test. Journal of Consulting Psychology 18, 6: 421-424. DOI: 10.1037/h0059972
Fred D. Davis, Richard P. Bagozzi, and Paul R. Warshaw. 1989. User acceptance of computer technology: A comparison of two theoretical models. Management Science 35, 8: 982-1003.
Efstathios Demitriades, Eastern Macedonia, and Dimitrios Chatzoudes. 2019. Factors Afecting the Acceptance of Electronic Books by the Readers. International Journal of Business and Economic Sciences Applied Research (IJBESAR) 12, 1: 7-19. DOI: 10.25103/ijbesar.121.01
Hermann Z. Ebbinghaus. 1897. Uber eine neue methode zur prüfung geistiger fähigkeiten and ihre anwendung bei schulkindern [About a new method for testing intellectual abilities]. Psychologie und Physiologie der Sinnesorgane 13: 401-457.
Chrysoula Gatsou, Anastasios Politis, and Dimitrios Zevgolis. 2015. Exploring users' experience with e-reading devices. Proceedings of the 2015 Federated Conference on Computer Science and Information Systems, FedCSIS 2015, IEEE, 833-839, DOI: 10.15439/2015F197.
Jin Gerlach and Peter Buxmann. 2011. Investigating the acceptance of electronic books -The impact of haptic dissonance on innovation adoption. 19th European Conference on Information Systems, ECIS 2011.
Jin Gerlach and Peter Buxmann. 2013. Analyzing electronic book acceptance: A compatibility perspective. Proceedings of the Annual Hawaii International Conference on System Sciences, IEEE, 2813-2822, DOI: 10.1109/HICSS.2013.94.
Chris Gibson and Forbes Gibb. 2011. An evaluation of second-generation ebook readers. Electronic Library 29, 3: 303-319. DOI: 10.1108/02640471111141061
Maaike S. Goelema, Renske De Bruijn, Sebastiaan Overeem, Els Møst, Reinder Haakma, and Panos Markopoulos. 2018. Conceptions of sleep experience: A layman perspective. BMC Research Notes 11(1): 1-5. DOI: 10.1186/s13104-0183584-2
Philip A. Goldberg. 1965. A Review of Sentence Completion Methods in Personality Assessment. Journal of Projective Techniques & Personality Assessment 29, 1: 12-45. DOI: 10.1080/0091651X.1965.10120175
Joy Paul Guilford. 1967. Creativity: Yesterday, Today and Tomorrow. The Journal of Creative Behavior 1, 1: 3-14. DOI: 10.1002/j.2162-6057.1967.tb00002.x
Steve Harrison and Deborah Tatar. 2011. On methods. Interactions, 10-11.
Marc Hassenzahl. 2003. The Thing and I: Understanding the Relationship Between User and Product. In M.A. Blythe, A.F. Monk, K. Overbeeke, and P.C. Wright, eds., Funology: From Usability to Enjoyment. Kluwer Academic Publishers.
Marc Hassenzahl, Michael Burmester, and Franz Koller. 2003. AttrakDif: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischerQualität. 187-196. DOI: 10.1007/978-3-322-80058-9-19
Margot Holaday, Debra A. Smith, and Alissa Sherry. 2000. Sentence completion tests: A review of the literature and results of a survey of members of the Society for Personality Assessment. Journal of Personality Assessment 74, 3: 371-383. DOI: 10.1207/S15327752JPA7403-3
Kai Michael Hover and Max Muhlhauser. 2014. The creation and evaluation of lecture E-books. Proceedings -IEEE 14th International Conference on Advanced Learning Technologies, ICALT 2014, IEEE, 647-649, DOI: 10.1109/ICALT.2014.190.
Wayne D. Hoyer and Deborah J. MacInnis. Consumer behavior (4th ed.). Houghton Mifin, New York, NY.
IFOP. 2014. Etude des perceptions et usages du livre numérique -Etude qualitative.
Evangelos Karapanos, Jean Bernard Martens, and Marc Hassenzahl. 2012. Reconstructing experiences with iScale. International Journal of Human Computer Studies 70, 11: 849-865. DOI: 10.1016/j.ijhcs.2012.06.004
Pekka Ketola and Virpi Roto. 2008. Exploring User Experience Measurement Needs. Proceedings of the 5th COST294-MAUSE International Workshop on Meaningful Measures Valid Useful User Experience Measurement VUUM, 23-26.
Sari Kujala and Piia Nurkka. 2009. Identifying user values for an activating game for children. MindTrek 2009 -13th International Academic MindTrek Conference: Everyday Life in the Ubiquitous Era, Association for Computing Machinery New York, NY, USA, 98-105, DOI: 10.1145/1621841.1621860.
Sari Kujala and Piia Nurkka. 2012. Sentence completion for evaluating symbolic meaning. International Journal of Design 6, 3: 15-25.
Sari Kujala, Tanja Walsh, Piia Nurkka, and Marian Crisan. 2014. Sentence completion for understanding users and evaluating user experience. Interacting with Computers 26, 3: 238-255. DOI: 10.1093/iwc/iwt036
Carine Lallemand. 2015. Towards Consolidated Methods for the Design and Evaluation of User Experience.
Carine Lallemand and avec Guillaume Gronier. 2018. Méthodes De design UX: 30 méthodes fondamentales pour concevoir des expériences optimales. Eyrolles, Paris.
Carine Lallemand and Vincent Koenig. 2017. Lab Testing Beyond Usability: Challenges and Recommendations for Assessing User Experiences. Journal of Usability Studies 12, 3: 133-154.
Carine Lallemand and Vincent Koenig. 2017. "How could an intranet be like a friend to me?" -Why standardized UX scales don't always ft. Proceedings of ACM ECCE conference (ECCE '17), Association for Computing Machinery New York, NY, USA, 9-16, DOI: 10.1145/3121283.3121288.
Bettina Laugwitz, Theo Held, and Martin Schrepp. 2008. Construction and evaluation of a user experience questionnaire. In A. Holzinger, ed., Lecture Notes in Computer Science. Springer-Verlag, Berlin Heidelberg, 63-76. DOI: 10.1007/978-3540-89350-9-6
Jonathan Lazar, Jinjuan Heidi Feng, and Harry Hochheiser. 2017. Research Methods in Human-Computer Interaction. Morgan Kaufmann Publishers, Cambridge, MA, USA.
Sascha Mahlke. 2007. User Experience of Interaction with Technical Systems. Retrieved from http://opus.kobv.de/tuberlin/volltexte/2008/1783/
Julian Meltzof. 1951. The efect of mental set and item structure upon response to a projective test. The Journal of Abnormal and Social Psychology 46, 2: 177-189. DOI: 10.1037/h0058624
Daphne Menheere, Mathias Funk, Erik Van Der Spek, Carine Lallemand, and Steven Vos. 2020.ADiary Study on the Exercise Intention-Behaviour Gap: Implica-tions for the Design of Interactive Products. Proceedings of DRS 2020 International Conference, Design Research Society London, UK, DOI: 10.21606/drs.2020.329.
Michael Minge, Manfred Thüring, Ingmar Wagner, and Carina V. Kuhr. 2016. The meCUE Questionnaire: A Modular Tool for Measuring User Experience. In M. Soares, C. Falcao, and T.Z. Ahram, eds., Advances in Ergonomics Modeling, Usability & Special Populations. Proceedings of the 7th Applied Human Factors and Ergonomics Society Conference 2016. Springer International Press, Switzerland, 115-128. DOI: 10.1007/978-3-319-41685-4-11
Bernard I. Murstein, Robert M. Colon, Nicole Destexhe-de Leval, and Martine Hoof-Van Van Parys. 1972. Infuence of Stimulus Properties of the Sentence-Completion-Method on Projection and Adjustment. Journal of Personality Assessment 36, 3: 241-247. DOI: 10.1080/00223891.1972.10119751
Eunil Park, Jungyeon Sung, and Kwangsu Cho. 2015. Reading experiences infuencing the acceptance of e-book devices. Electronic Library 33, 1: 120-135. DOI: 10.1108/EL-05-2012-0045
Robert F. Peck and MC Guire Carson. 1959. Measuring Changes in Mental Health with the Sentence Completion Technique. Psychological Reports 5, 1: 151-160. DOI: 10.2466/pr0.1959.5.g.151
Ingrid Pettersson, Florian Lachner, Anna Katharina Frison, Andreas Riener, and Andreas Butz. 2018. A bermuda triangle? -A review of method application and triangulation in user experience evaluation. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, Association for Computing Machinery New York, NY, USA, DOI: 10.1145/3173574.3174035.
Kenneth E. Rogers, John Bishop, and Robert C. Lane. 2003. Considerations for the Use of Sentence Completion Tests. Journal of Contemporary Psychotherapy33, 3: 235-242. DOI: 10.1023/A:1023977704349
Amanda R. Rohde. 1957. The Sentence Completion Method: Its Diagnostic and Clinical Application to Mental Disorders. The Ronald Press Company, New York, NY, USA.
Björn Rohles, Vincent Koenig, Antoine Fischbach, and Franck Amadieu. 2019. Experience matters: Bridging the gap between experience-and functionality-driven design in technology-enhanced learning. Interaction Design and Architecture(s) 42: 11-28.
Virpi Roto, Efe Law, Arnold Vermeeren, and Jettie Hoonhout. 2011. User Experience White Paper: Bringing clarity to the concept of user experience.
Mark A. Runco and Garrett J. Jaeger. 2012. The Standard Defnition of Creativity. Creativity Research Journal 24, 1: 92-96. DOI: 10.1080/10400419.2012.650092
Joseph M. Sacks. 1949. The relative efect upon projective responses of stimuli referring to the subject and of stimuli referring to other persons. Journal of Consulting Psychology 13, 1: 12-20. DOI: 10.1037/h0055829
Jami J. Shah, Noe Vargas-Hernandez, and Steve M. Smith. 2003. Metrics for measuring ideation efectiveness. Design Studies 24, 2: 111-134. DOI: 10.1016/S0142694X(02)00034-0
Kennon M. Sheldon, Andrew J. Elliot, Youngmee Kim, and Tim Kasser. 2001. What is satisfying about satisfying events? Testing 10 candidate psychological needs. Journal of Personality and Social Psychology 80, 2: 325-339. DOI: 10.1037/00223514.80.2.325
Chaehan So and Jaewoo Joo. 2017. Does a Persona Improve Creativity? The Design Journal 20, 4: 459-475. DOI: 10.1080/14606925.2017.1319672
SOFIA, SNE, and SGD. 2016. 6e baromètre sur les usages du livre numérique Un marché stable qui confrme la pratique d 'un usage mixte: Livres numériques et imprimés coexistent.
Sofa, SNE, and SGDL. 2014. Baromètre des usages du livre numérique -vague 4.
Laurence C. Soley and Aaron Lee Smith. 2008. Projective techniques for social science and business research. Southshore Press, Milwaukee, WI, USA.
Mark W. Stephens. 1970. Stimulus Pull as a Determinant of Individual Diferences in Sentence Completion Responses. Journal of Projective Techniques and Personality Assessment 34, 4: 332-339. DOI: 10.1080/0091651X.1970.10380258
Erik Stolterman. 2008. The nature of design practice and implications for interaction design research. International Journal of Design 2, 1: 55-65.
David Swallow, Mark Blythe, and Peter Wright. 2005. Grounding Experience: Relating Theory and Method to Evaluate the User Experience of Smartphones. Proceedings of the 2005 annual conference on European association of cognitive ergonomics (EACE '05), Association for Computing Machinery New York, NY, USA, 91-98.
Esta Tovstiadi, Natalia Tingle, and Gabrielle Wiersma. 2018. Academic E-book Usability from the Student's Perspective. Evidence Based Library and Information Practice 13, 4: 70-87. DOI: 10.18438/eblip29457
David K. Trites. 1956. Evaluation of assumptions underlying interpretation of sentence completion tests. Journal of Consulting Psychology 20, 1: 8. DOI: 10.1037/h0044329
Wen Chia Tsai. 2012. A study of consumer behavioral intention to use e-books: The Technology Acceptance Model perspective. Innovative Marketing 8, 4: 55-66.
Wenn Chieh Tsai and Elise Van Den Hoven. 2018. Memory probes: Exploring retrospective user experience through traces of use on cherished objects. Inter-national Journal of Design 12, 3: 57-72.
Karen Turnbow and Richard H. Dana. 1981. The Efects of Stem Length and Directions on Sentence Completion Test Responses. Journal of Personality Assessment 45, 1: 27-32. DOI: 10.1207/s15327752jpa4501-7
Heli Kristiina Väätäjä, Paula Savioja, Virpi Roto, Thomas Daniel Olsson, and Jari Tapio Varsaluoma. 2015. User experience goals as a guiding light in design and development? Early fndings. INTERACT 2015 Adjunct proceedings, University of Bamberg Press, 521-527.
Arnold P.O.S. Vermeeren, Efe Lai Chong Law, Virpi Roto, Marianna Obrist, Jettie Hoonhout, and Kaisa Väänänen-Vainio-Mattila. 2010. User experience evalua-tion methods: Current state and development needs. NordiCHI 2010: Extending Boundaries -Proceedings of the 6th Nordic Conference on Human-Computer In-teraction, Association for Computing Machinery New York, NY, USA, 521-530, DOI: 10.1145/1868914.1868973.
Tanja Walsh, Piia Nurkka, and Sari Kujala. 2010. Exploring Cross-Cultural UX Evaluation with the Remote Online Sentence Completion Method Cross-Cultural Design. Proceedings of the Ninth International Workshop on Internationalisation of Products and Systems IWIPS 2010, London, UK, 11-22.
Tanja Walsh, Piia Nurkka, Helen Petrie, and Jaana Olsson. 2013. The efect of language in answering qualitative questions in user experience evaluation web-surveys. Proceedings of the 25th Australian Computer-Human Interaction Conference: Augmentation, Application, Innovation, Collaboration, OzCHI 2013, Association for Computing Machinery New York, NY, USA, 73-82, DOI: 10.1145/2541016.2541049.
Tanja Walsh, Piia Nurkka, and Rod Walsh. 2010. Cultural diferences in smartphone user experience evaluation. Proceedings of the 9th International Conference on Mobile and Ubiquitous Multimedia (MUM '10), Association for Computing Machinery New York, NY, USA, 1-9, DOI: 10.1145/1899475.1899499.
Charles A. Weisgerber. 1969. Psychological assessment of candidates for a religious order. Loyola University Press, Chicago.
Yuanxiang Zeng, Xue Bai, Jie Xu, and Chuqi Gong Huan He. 2016. The Infuence of E-book Format and Reading Device on Users' Reading Experience: A Case Study of Graduate Students. Publishing Research Quarterly 32, 4: 319-330. DOI: 10.1007/s12109-016-9472-5