Evaluating ambiguity of privacy indicators in a secure email app

Informing laymen of security situations is a notoriously hard problem. Users are usually not cognoscenti of all the various secure and insecure situations that may arise, and this can be further worsened by certain visual indicators that instead of helping users, fail to convey clear and unambiguous messages. Even in well-established and studied applications, like email clients providing end-to-end encryption, the problem seems far from being solved. Motivated to verify this claim, we studied the communication qualities of four privacy icons (in the form of coloured shapes) in conveying specific security messages, relevant for a particular secure emailing system called p≡p. We questioned 42 users in three different sessions, where we showed them 10 privacy ratings, along with their explanations, and asked them to match the rating and explanation with the four privacy icons. We compared the participants’ associations to those made by the p≡p developers.
The results, still preliminary, are not encouraging. Except for the two most extreme cases, Secure and trusted and Under attack, users almost entirely missed to get the indicators’ intended messages. In particular, they did not grasp certain concepts such as Unsecure email and Secure email, which in turn were fundamental for the engineers. Our work has certain limitations and further investigation is required, but already at this stage our research calls for a closer collaboration between app engineers and icon designers. In the context of p≡p, our work has triggered a deeper discussion on the icon design choices and a potential revamp is on the way.