How Could an Intranet be Like a Friend to Me?: Why Standardized UX Scales Don't Always Fit

LALLEMAND, Carine; KOENIG, Vincent

doi:10.1145/3121283.3121288

Download

Paper published in a journal (Scientific congresses, symposiums and conference proceedings)

How Could an Intranet be Like a Friend to Me?: Why Standardized UX Scales Don't Always Fit

LALLEMAND, Carine; KOENIG, Vincent

2017 • In Proceedings of the European Conference on Cognitive Ergonomics 2017, p. 9-16

Peer reviewed

Permalink
https://hdl.handle.net/10993/32791

DOI
10.1145/3121283.3121288

Files (1)Send to Details Statistics Bibliography Similar publications

Files

Full Text

p9-lallemand.pdf

Publisher postprint (201.59 kB)

Download

All documents in ORBilu are protected by a user license.

Send to

RIS BibTex APA Chicago Permalink X Linkedin

Details

Keywords :

Standardized UX scales; meCUE questionnaire; workplace UX assessment; UX; user experience; evaluation; intranet; design

Abstract :

[en] "I hope that this survey is a joke because it made me laugh so much". This quote is just one example of many negative respondents' reactions gathered during a large-scale user experience (UX) study. Unfortunately, the survey was no joke, rather a well-constructed and validated standardized UX scale. This paper critically reflects on the use and relevance of standardized UX scales for the evaluation of UX in business contexts. We report on a real-world use case where the meCUE questionnaire has been used to assess employees' experience (N=263) with their organization's intranet. Strong users' reactions to the survey's items and statistical analyses both suggest that the scale is unsuitable for the evaluation of business-oriented systems. Drawing on the description of this inadequacy, we discuss the quality of academic UX tools, calling into question the relevance for practice of academic methods.

Disciplines :

Social & behavioral sciences, psychology: Multidisciplinary, general & others

Author, co-author :

LALLEMAND, Carine ; University of Luxembourg > Faculty of Language and Literature, Humanities, Arts and Education (FLSHASE) > Education, Culture, Cognition and Society (ECCS)

KOENIG, Vincent ; University of Luxembourg > Faculty of Language and Literature, Humanities, Arts and Education (FLSHASE) > Education, Culture, Cognition and Society (ECCS)

External co-authors :

Language :

English

Title :

How Could an Intranet be Like a Friend to Me?: Why Standardized UX Scales Don't Always Fit

Publication date :

October 2017

Event name :

European Conference on Cognitive Ergonomics 2017

Event place :

Umeå, Sweden

Event date :

September 19 - 22, 2017

Audience :

International

Journal title :

Proceedings of the European Conference on Cognitive Ergonomics 2017

Publisher :

ACM, New York, United States

Pages :

9-16

Peer reviewed :

Peer reviewed

Additional URL :

http://dl.acm.org/citation.cfm?id=3121288&CFID=970440474&CFTOKEN=58576616

Available on ORBilu :

since 03 November 2017

Statistics

Number of views

275 (15 by Unilu)

Number of downloads

614 (7 by Unilu)

More statistics

Scopus citations^®

Scopus citations^®
without self-citations

OpenCitations

OpenAlex citations

Bibliography

R. Alves, P. Valente, and N. J. Nunes. 2014. The state of user experience evaluation practice. In Proc. of NordiCHI'14. New York, NY: ACM Press (pp. 93-102). doi:10.1145/2639189.2641208
Regina Bernhaupt and Michael Pirker. 2013. Evaluating User Experience for Interactive Television: Towards the Development of a Domain-Specific User Experience Questionnaire. IFIP TC13 Conference on Human-Computer Interaction INTERACT'13. Lecture Notes in Computer Science 8118, 642-659. Berlin/Heidelberg: Springer.
J. H. Brockmyer, C. M. Fox, K. A. Curtiss, E. McBroon, K. M. Burkhart, and J. N. Pidruzny. 2009. The development of the Game Engagement Questionnaire: A measure of engagement in video game-playing. Journal of Experimental Social Psychology, 45, 624-634.
John Brooke. 1996. SUS: a "quick and dirty" usability scale. In P. W. Jordan, B. Thomas, B. A. Weerdmeester, & A. L. McClelland (Eds.). Usability Evaluation in Industry. London: Taylor and Francis.
Paul Cairns. 2007. HCI⋯ Not As It Should Be: Inferential Statistics in HCI Research. Proceedings of the 21st British HCI Group Annual Conference on People and Computers BCS-HCI '07. Swinton, UK: British Computer Society.
D. B. Chertoff, B. Goldiez, and J. J. LaViola. 2010. Virtual Experience Test: A virtual environment evaluation questionnaire. Proceedings of the 2010 IEEE Virtual Reality Conference VR'10. Washington, DC, USA: IEEE Computer Society.
Alan Dix. 2010. Human-Computer Interaction: a stable discipline, a nascent science, and the growth of the long tail. Interacting with Computers, 22(1), 13-27.
C. M. Gray, E. Stolterman and M. A. Siegel. 2014. Reprioritizing the Relationship Between HCI Research and Practice: Bubble-Up and Trickle-Down Effects. Proceedings of the 2014 conference on Designing interactive systems DIS'14. New York, NY: ACM Press. doi: 10.1145/2598510.2598595
M. Hassenzahl, M. Burmester, and F. Koller. 2003. AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualität. In J. Ziegler, & G. Szwillus (Eds.), Mensch & Computer 2003. Interaktion in Bewegung (pp. 187-196). Stuttgart, Germany: B.G. Teubner. doi:10.1007/978-3-322-80058-9-19
Jurek Kirakowski and Mary Corbett. 1993. SUMI: The Software Usability Measurement Inventory. British Journal of Educational Technology, 24, 210-212.
Carina Kuhr. 2013. Measuring the User Experience of Mobile Applications-an Empirical Validation of a Quantitative Method. (Master thesis). Technische Universität Berlin. http://mecue.de/Homepage%20Content/02%20Links%20%26%20Literatur/MA-Kuhr.pdf
Carine Lallemand. 2015. Towards Consolidated Methods for the Design and Evaluation of User Experience. (Doctoral dissertation). University of Luxembourg. https://publications.uni.lu/handle/10993/21463
C. Lallemand, V. Koenig, G. Gronier, and R. Martin. 2015. Création et validation d'une version française du questionnaire AttrakDiff pour l'évaluation de l'expérience utilisateur des systèmes interactifs. Revue européenne de psychologie appliquée, 65(5), 239-252. doi:10.1016/j.erap.2015.08.002
B. Laugwitz, T. Held, and M. Schrepp. 2008. Construction and Evaluation of a User Experience Questionnaire. In A. Holzinger (Ed.): HCI and Usability for Education and Work USAB 2008, LNCS 5298, pp. 63-76. Berlin/Heidelberg: Springer.
M. Minge, M. Thüring, and I. Wagner. 2016. Developing and Validating an English Version of the meCUE Questionnaire for Measuring User Experience. In Proc. of the HFES'16. doi: http://dx.doi.org/10.1177/1541931213601468
Michael Minge and Laura Riedel. 2013. meCUE-Ein modularer Fragebogen zur Erfassung des Nutzungserlebens. In: S. Boll, S. Maaß & R. Malaka (Hrsg.): Mensch und Computer 2013: Interaktive Vielfalt (S. 89-98). München, Oldenbourg Verlag.
Paul E. Spector. 1992. Summated rating scale construction. An Introduction. Newbury Park: Sage.
Manfred Thüring and Sascha Mahlke. 2007. Usability, aesthetics and emotions in human-technology interaction. International Journal of Psychology, 42(4), 253-264. doi:10.1080/00207590701396674
Tom Tullis and William Albert. 2013. Measuring the User Experience (2nd ed.): Collecting, Analyzing, and Presenting Usability Metrics. Burlington, MA: Morgan Kaufmann Publishers.
K. Väänänen-Vainio-Mattila, V. Roto, and M. Hassenzahl. 2008. Towards practical user experience evaluation methods. Proceedings of Meaningful Measures: Valid Useful User Experience Measurement (VUUM), 5th COST294-MAUSE Open Workshop, 18th June 2008, Reykjavik, Iceland.
Kaisa Väänänen-Vainio-Mattila and Minna Wäljas. 2009. Development of Evaluation Heuristics for Web Service User Experience. Extended Abstracts on Human Factors in Computing Systems (CHI EA'09). New York, NY, USA: ACM Press. doi: 10.1145/1520340.1520554
Robert J. Vallerand. 1989. Vers une méthodologie de validation transculturelle de questionnaires psychologiques: implications pour la recherche en langue française. Psychologie Canadienne, 30(4), 662-689.
Fons Van de Vijver and Ronald K. Hambleton. 1996. Translating tests: Some practical guidelines. European Psychologist, 1, 89-99.
Fons Van de Vijver and Norbert K. Tanzer. 2004. Bias and equivalence in cross-cultural assessment: An overview. European Review of Applied Psychology, 54, 119-135. doi: 10.1016/j.erap.2003.12.004
A. Vermeeren, E. Law, V. Roto, M. Obrist, J. Hoonhout and K. Väänänen-Vainio-Mattila. 2010. User Experience Evaluation Methods: Current State and Development Needs. Proceedings of NordiCHI 2010, New York, NY: ACM Press. doi:10.1145/1868914.1868973