dark patterns; online manipulation; digital nudging; consumer protection; user experience; user interface
Abstract :
[en] Online services pervasively employ manipulative designs (i.e., dark patterns) to influence users to purchase goods and subscriptions, spend more time on-site, or mindlessly accept the harvesting of their personal data. To protect users from the lure of such designs, we asked: are users aware of the presence of dark patterns? If so, are they able to resist them? By surveying 406 individuals, we found that they are generally aware of the influence that manipulative designs can exert on their online behaviour. However, being aware does not equip users with the ability to oppose such influence. We further find that respondents, especially younger ones, often recognise the "darkness" of certain designs, but remain unsure of the actual harm they may suffer. Finally, we discuss a set of interventions (e.g., bright patterns, design frictions, training games, applications to expedite legal enforcement) in the light of our findings.
Disciplines :
Social & behavioral sciences, psychology: Multidisciplinary, general & others
Author, co-author :
Bongard-Blanchy, Kerstin ; University of Luxembourg > Faculty of Humanities, Education and Social Sciences (FHSE) > Department of Behavioural and Cognitive Sciences (DBCS)
Rossi, Arianna ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > IRiSC
Rivas, Salvador ; University of Luxembourg > Faculty of Humanities, Education and Social Sciences (FHSE) > LUCET
Doublet, Sophie ; University of Luxembourg > Faculty of Humanities, Education and Social Sciences (FHSE) > Department of Behavioural and Cognitive Sciences (DBCS)
Koenig, Vincent ; University of Luxembourg > Faculty of Humanities, Education and Social Sciences (FHSE) > Department of Behavioural and Cognitive Sciences (DBCS)
Lenzini, Gabriele ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > IRiSC
External co-authors :
no
Language :
English
Title :
"I am definitely manipulated, even when I am aware of it. It’s ridiculous!" - Dark Patterns from the End-User Perspective
Publication date :
2021
Event name :
DIS Conference on Designing Interactive Systems
Event date :
June 28 - July 2, 2021
Main work title :
Proceedings of ACM DIS Conference on Designing Interactive Systems
Alessandro Acquisti, Idris Adjerid, Rebecca Balebako, Laura Brandimarte, Lorrie Faith Cranor, Saranga Komanduri, Pedro Giovanni Leon, Norman Sadeh, Florian Schaub, Manya Sleeper, et al. 2017. Nudges for privacy and security: Understanding and assisting users' choices online. ACM Computing Surveys (CSUR) 50, 3 (2017), 1-41.
Devdatta Akhawe and Adrienne Porter Felt. 2013. Alice inWarningland: A Large-Scale Field Study of Browser Security Warning Effectiveness. In 22nd USENIX Security Symposium (USENIX Security 13). USENIX Association, Washington, D.C., 257-272. https://www.usenix.org/conference/usenixsecurity13/technicalsessions/presentation/akhawe
Adam Alter. 2017. Irresistible: The rise of addictive technology and the business of keeping us hooked. Penguin Publishing Group.
Susanne Barth and Menno DT De Jong. 2017. The privacy paradox-Investigating discrepancies between expressed privacy concerns and actual online behavior-A systematic literature review. Telematics and informatics 34, 7 (2017), 1038-1058.
Christoph Bösch, Benjamin Erb, Frank Kargl, Henning Kopp, and Stefan Pfattheicher. 2016. Tales from the dark side: Privacy dark strategies and privacy dark patterns. Proceedings on Privacy Enhancing Technologies 2016, 4 (2016), 237-254.
David M. Boush, Marian Friestad, and Peter Wright. 2009. Deception In The Marketplace: The Psychology of Deceptive Persuasion and consumer self-protection (first edition ed.). Rouledge.
Nadia M. Brashier and Daniel L. Schacter. 2020. Aging in an Era of Fake News. Current Directions in Psychological Science 29, 3 (Jun 2020), 316-323. https://doi.org/10.1177/0963721420915872
Cristian Bravo-Lillo, Lorrie Cranor, Saranga Komanduri, Stuart Schechter, and Manya Sleeper. 2014. Harder to ignore? Revisiting pop-up fatigue and approaches to prevent it. In 10th Symposium On Usable Privacy and Security ({SOUPS} 2014). 105-111.
Harry Brignull. 2010. Dark Patterns. https://darkpatterns.org/
John Brownlee. 2016. Why Dark Patterns Won't Go Away. https://www.fastcompany.com/3060553/why-dark-patterns-wont-go-away
Régis Chatellier, Geoffrey Delcroix, Estelle Hary, and Camille Girard-Chanudet. 2019. Shaping choices in the digital world. From dark patterns to data protection: the influence of ux/ui design on user empowerment. Technical Report. CNIL.
Shruthi Sai Chivukula and Colin M GRAY. 2020. Co-Evolving Towards Evil Design Outcomes: Mapping Problem and Solution Process Moves. In Proceedings of the Design Research Society Conference.
Shruthi Sai Chivukula, Colin M Gray, and Jason A Brier. 2019. Analyzing value discovery in design decisions through ethicography. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1-12.
Shruthi Sai Chivukula, Chris Rhys Watkins, Rhea Manocha, Jingle Chen, and Colin M Gray. 2020. Dimensions of UX Practice that Shape Ethical Awareness. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1-13.
CNIL. [n.d.]. Données & Design. Co-building user journeys compliant with the GDPR and respectful of privacy. https://design.cnil.fr/
CNIL. 2019. Deliberation of the Restricted Committee SAN-2019-001 of 21 January 2019 pronouncing a financial sanction against GOOGLE LLC. https://www.cnil. fr/sites/default/files/atoms/files/san-2019-001.pdf
CNIL. 2020. Cookies: financial penalties of 60 million euros against the company GOOGLE LLC and of 40 million euros against the company GOOGLE IRELAND LIMITED. https://www.cnil.fr/en/cookies-financial-penalties-60-million-eurosagainst-company-google-llc-and-40-million-euros-google-ireland
Gregory Conti and Edward Sobiesk. 2010. Malicious interface design: exploiting the user. In Proceedings of the 19th international conference on World wide web-WWW '10. ACM Press, 271. https://doi.org/10.1145/1772690.1772719
Anna L. Cox, Sandy J.J. Gould, Marta E. Cecchinato, Ioanna Iacovides, and Ian Renfree. 2016. Design Frictions for Mindful Interactions: The Case for Microboundaries. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '16). Association for Computing Machinery, 1389-1397. https://doi.org/10.1145/2851581.2892410
Linda Di Geronimo, Larissa Braz, Enrico Fregnan, Fabio Palomba, and Alberto Bacchelli. 2020. UI dark patterns and where to find them: a study on mobile applications and user perception. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1-14.
Verena Distler, Gabriele Lenzini, Carine Lallemand, and Vincent Koenig. 2020. The Framework of Security-Enhancing Friction: How UX Can Help Users Behave More Securely. In New Security Paradigms Workshop 2020. ACM, 45-58. https://doi.org/10.1145/3442167.3442173
EDPB. 2020. Guidelines 05/2020 on consent under Regulation 2016/679.
Trine Falbe, Kim Andersen, and Martin Michael Frederiksen. 2020. The ethical design handbook. Smashing Media AG.
Forbrukerrådet. 2021. You can log out, but you can never leave. How Amazon manipulates consumers to keep them subscribed to Amazon prime. https://fil.forbrukerradet.no/wp-content/uploads/2021/01/2021-01-14-you-can-log-out-but-you-can-never-leave-final.pdf
Colin M. Gray, Jingle Chen, Shruthi Sai Chivukula, and Liyang Qu. 2020. End User Accounts of Dark Patterns as Felt Manipulation. arXiv: 2010.11046 [cs] (Oct 2020). http://arxiv.org/abs/2010.11046 arXiv: 2010.11046.
Colin M Gray and Shruthi Sai Chivukula. 2019. Ethical mediation in UX practice. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1-11.
Colin M Gray, Shruthi Sai Chivukula, and Ahreum Lee. 2020. What Kind ofWork Do" Asshole Designers" Create? Describing Properties of Ethical Concern on Reddit. In Proceedings of the 2020 ACM on Designing Interactive Systems Conference. 61-73.
Colin M Gray, Yubo Kou, Bryan Battles, Joseph Hoggatt, and Austin L Toombs. 2018. The dark (patterns) side of UX design. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1-14.
Paul Graßl, Hanna Schraffenberger, Frederik Zuiderveen Borgesius, and Moniek Buijzen. 2021. Dark and Bright Patterns in Cookie Consent Requests. Journal of Digital Social Research 3, 11 (Feb 2021), 1-38. https://doi.org/10.33621/jdsr.v3i1.54
Saul Greenberg, Sebastian Boring, Jo Vermeulen, and Jakub Dostal. 2014. Dark patterns in proxemic interactions: a critical perspective. In Proceedings of the 2014 conference on Designing interactive systems. 523-532.
Pelle Guldborg Hansen and Andreas Maaløe Jespersen. 2013. Nudge and the manipulation of choice: A framework for the responsible use of the nudge approach to behaviour change in public policy. European Journal of Risk Regulation 4, 1 (2013), 3-28.
Ralph Hertwig and Till Grüne-Yanoff. 2017. Nudging and Boosting: Steering or Empowering Good Decisions. Perspectives on Psychological Science 12, 6 (Nov 2017), 973-986. https://doi.org/10.1177/1745691617702496
Soheil Human and Florian Cech. 2020. A Human-centric Perspective on Digital Consenting: The Case of GAFAM. In Human Centred Intelligent Systems. Springer, 139-159.
Arushi Jaiswal. 2018. Dark patterns in UX: how designers should be responsible for their actions. https://uxdesign.cc/dark-patterns-in-ux-design-7009a83b233c
Georgios Kampanos and Siamak F. Shahandashti. 2021. Accept All: The Landscape of Cookie Banners in Greece and the UK. arXiv: 2104.05750 [cs] (Apr 2021). http://arxiv.org/abs/2104.05750 arXiv: 2104.05750.
Lexie Kane. 2019. The Attention Economy. https://www.nngroup.com/articles/attention-economy/
Julie A. Kientz, Eun Kyoung Choe, Brennen Birch, Robert Maharaj, Amanda Fonville, Chelsey Glasson, and Jen Mundt. 2010. Heuristic evaluation of persuasive health technologies. In Proceedings of the ACM international conference on Health informatics-IHI '10. ACM Press, 555. https://doi.org/10.1145/1882992. 1883084
UXP2 Lab. 2018. The dark side of UX Design. https://darkpatterns.uxp2.com/
C. Lacey and C. Caudwell. 2019. Cuteness as a 'Dark Pattern' in Home Robots. In 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI). 374-381. https://doi.org/10.1109/HRI.2019.8673274
M. R. Leiser. 2020. "Dark Patterns": The Case for Regulatory Pluralism. SSRN Repository (Jun 2020). https://doi.org/10.2139/ssrn.3625637 DOI: 10.2139/ssrn.3625637.
Tim-Benjamin Lembcke, Nils Engelbrecht, Alfred Benedikt Brendel, Bernd Herrenkind, and Lutz M. Kolbe. 2019. Towards a Unified Understanding of Digital Nudging by Addressing its Analog Roots. In PACIS.
Marco Lippi, Przemyslaw Palka, Giuseppe Contissa, Francesca Lagioia, Hans-Wolfgang Micklitz, Giovanni Sartor, and Paolo Torroni. 2019. CLAUDETTE: an automated detector of potentially unfair clauses in online terms of service. Artificial Intelligence and Law 27, 2 (2019), 117-139.
Jamie Luguri and Lior Strahilevitz. 2019. Shining a light on dark patterns. U of Chicago, Public Law Working Paper 719 (2019).
Aditi M. Bhoot, Mayuri A. Shinde, and Wricha P. Mishra. 2020. Towards the Identification of Dark Patterns: An Analysis Based on End-User Reactions. In IndiaHCI '20: Proceedings of the 11th Indian Conference on Human-Computer Interaction (IndiaHCI 2020). Association for Computing Machinery, 24-33. https://doi.org/10.1145/3429290.3429293
Dominique Machuletz and Rainer Böhme. 2020. Multiple purposes, multiple problems: A user study of consent dialogs after GDPR. Proceedings on Privacy Enhancing Technologies 2020, 2 (2020), 481-498.
Maximilian Maier and Rikard Harr. 2020. Dark Design Patterns: An End-user Perspective. Human Technology 16, 2 (2020), 170-199.
Arunesh Mathur, Gunes Acar, Michael J Friedman, Elena Lucherini, Jonathan Mayer, Marshini Chetty, and Arvind Narayanan. 2019. Dark patterns at scale: Findings from a crawl of 11K shopping websites. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 1-32.
Arunesh Mathur, Jonathan Mayer, and Mihir Kshirsagar. 2021. What Makes a Dark Pattern⋯ Dark? Design Attributes, Normative Considerations, and Measurement Methods. arXiv: 2101.04843 [cs] (Jan 2021). https://doi.org/10.1145/3411764. 3445610 arXiv: 2101.04843.
Arunesh Mathur, AngelinaWang, Carsten Schwemmer, Maia Hamin, Brandon M. Stewart, and Arvind Narayanan. 2020. Manipulative tactics are the norm in political emails: Evidence from 100K emails from the 2020 U.S. election cycle. https://electionemails2020.org.
Célestin Matte, Nataliia Bielova, and Cristiana Santos. 2020. Do Cookie Banners Respect my Choice?: Measuring Legal Compliance of Banners from IAB Europe's Transparency and Consent Framework. In 2020 IEEE Symposium on Security and Privacy (SP). IEEE, 791-809.
Christian Meske and Ireti Amojo. 2020. Ethical Guidelines for the Construction of Digital Nudges. In 53rd Hawaii International Conference on Systems Sciences (HICSS). 3928-3937. http://arxiv.org/abs/2003.05249 arXiv: 2003.05249.
Sarah Theres Völkel Daniel Buschek Michael and Malin Eiband Chromik. 2019. Dark Patterns of Explainability, Transparency, and User Control for Intelligent Systems. In 2nd Workshop on Explainable Smart Systems at the ACM Conference on Intelligent User Interfaces (IUI'19).
Carol Moser, Sarita Y. Schoenebeck, and Paul Resnick. 2019. Impulse Buying: Design Practices and Consumer Needs. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems-CHI '19. ACM Press, 1-15. https://doi.org/10.1145/3290605.3300472
Netherlands Authority for Consumers & Markets. 2020. ACM Guidelines on the Protection of the Online Consumer. Boundaries of Online Persuasion. https://www.acm.nl/sites/default/files/documents/2020-02/acm-guidelineson-the-protection-of-the-online-consumer.pdf
Midas Nouwens, Ilaria Liccardi, Michael Veale, David Karger, and Lalana Kagal. 2020. Dark patterns after the GDPR: Scraping consent pop-ups and demonstrating their influence. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1-13.
Court of Justice of the European Union. 2019. C-673/17-Planet49. Judgment of the Court (Grand Chamber) of 1 October 2019 Bundesverband der Verbraucherzentralen und Verbraucherverbände-Verbraucherzentrale Bundesverband e.V. v Planet49 GmbH. http://curia.europa.eu/juris/liste.jsf?num=C-673/17
Information Commissioner Office. 2020. Age appropriate design: a code of practice for online services. https://ico.org.uk/for-organisations/guide-todata-protection/key-data-protection-themes/age-appropriate-design-a-codeof-practice-for-online-services/
Stigler Committee on Digital Platforms. 2019. "Final Report, " Stigler Center for the Study of the Economy and the State. (Sep 2019). https://research.chicagobooth . edu/stigler/media/news/committee-on-digital-platforms-final-report
Eyal Peer, Serge Egelman, Marian Harbach, Nathan Malkin, Arunesh Mathur, and Alisa Frik. 2020. Nudge Me Right: Personalizing Online Security Nudges to People's Decision-Making Styles. Computers in Human Behavior 109, August 2020, 106347 (2020). https://doi.org/10.1016/j.chb.2020.106347 DOI: 10.2139/ssrn.3324907.
Forbruker Radet. 2018. Deceived by design. How tech companies use dark patterns to discourage us from exercising our rights to privacy. Technical Report.
Ronald W. Rogers and Steven Prentice-Dunn. 1997. Protection motivation theory (d. s. gochman ed.). Plenum Press, 113-132.
Cristiana Santos, Nataliia Bielova, and Célestin Matte. 2020. Are cookie banners indeed compliant with the law?:. Technology and Regulation 2020 (Dec 2020), 91-135. https://doi.org/10.26116/techreg.2020.009
Tali Sharot. 2011. The optimism bias. Current biology 21, 23 (2011), R941-R945.
Simon Shaw. 2019. Consumers Are Becoming Wise to Your Nudge. https://behavioralscientist.org/consumers-are-becoming-wise-to-your-nudge/
Dwayne D. Sijun Wang, Sharon E. Beatty, and William Foxx. 2004. Signaling the Trustworthiness of Small Online Retailers. Journal of Interactive Marketing (John Wiley and Sons) 18, 1 (2004), 53-69. https://doi.org/10.1002/dir.10071
Natasha Singer. 2016. When Websites Won't Take No for an Answer. The New York Times (May 2016). https://www.nytimes.com/2016/05/15/technology/personaltech/when-websites-wont-take-no-for-an-answer.html
Than Htut Soe, Oda Elise Nordberg, Frode Guribye, and Marija Slavkovik. 2020. Circumvention by design-dark patterns in cookie consents for online news outlets. In NordiCHI 2020.
Daniel Susser, Beate Roessler, and Helen Nissenbaum. 2019. Technology, Autonomy, and Manipulation. https://papers.ssrn.com/abstract=3420747
Daniel Susser, Beate Roessler, and Helen F. Nissenbaum. 2018. Online Manipulation: Hidden Influences in a Digital World. Georgetown Law Technology Review 1 (2018), 1-45.
Unknown. 2019. Dark pattern games. https://www.darkpattern.games/
Christine Utz, Martin Degeling, Sascha Fahl, Florian Schaub, and Thorsten Holz. 2019. (Un) informed Consent: Studying GDPR Consent Notices in the Field. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security. 973-990.
Ari Ezra Waldman. 2020. Cognitive biases, dark patterns, and the 'privacy paradox'. Current opinion in psychology 31 (2020), 105-109.
Rick Wash. 2010. Folk Models of Home Computer Security. In Proceedings of the Sixth Symposium on Usable Privacy and Security (Redmond, Washington, USA) (SOUPS '10). Association for Computing Machinery, New York, NY, USA, Article 11, 16 pages. https://doi.org/10.1145/1837110.1837125
Christopher Rhys Watkins, Colin M Gray, Austin L Toombs, and Paul Parsons. 2020. Tensions in Enacting a Design Philosophy in UX Practice. In Proceedings of the 2020 ACM on Designing Interactive Systems Conference. 2107-2118.
RyanWest. 2008. The psychology of security. Commun. ACM 51, 4 (2008), 34-40.
José P Zagal, Staffan Björk, and Chris Lewis. 2013. Dark Patterns in the Design of Games. In Proceedings of the 8th International Conference on the Foundations of Digital Games (FDG 2013). 39-46.
