Why Johnny the Developer Can’t Work with Public Key Certificates

March 3, 2020

Research cooperation of CRoCS, Faculty of Informatics, Masaryk University and Red Hat.

Abstract

There have been many studies exposing poor usability of security software for the common end user. However, only a few inspect the usability challenges faced by more knowledgeable users. We conducted an experiment to empirically assess usability of the command line interface of OpenSSL, a well known and widely used cryptographic library. Based on the results, we try to propose specific improvements that would encourage more secure behavior. We observed 87 developers/administrators at two certificate-related tasks in a controlled environment. Furthermore, we collected participant opinions on both the tool interface and available documentation. Based on the overall results, we deem the OpenSSL usability insufficient according to both user opinions and standardized measures. Moreover, the perceived usability seems to be correlated with previous experience and used resources. There was a great disproportion between the participant views of a successful task accomplishment and the reality. A general dissatisfaction with both OpenSSL interface and its manual page was shared among the majority of the participants. As hinted by a participant, OpenSSL gradually “turned into a complicated set of sharp kitchen knives” – it can perform various jobs very well, but laymen risk stabbing themselves in the process. This highlights the necessity of a usable design even for tools targeted at experienced users.

Several interesting bits…

The usability of OpenSSL interface turned out to be… not so great (no real surprise there). But there are some interesting bits and pieces. To name just a few:

  • 87% participants thought they successfully issued a self-signed certificate, but only 45% did.
  • 42% created a certificate with “Internet Widgits Pty Ltd.” in an organization field.
  • 26% of created certificates were old version 1.
  • 73% participants used Stack Overflow solutions, 40% used the knowledge base of University of Wisconsin-Madison.
  • 28% people tried to open the manual page with ‘man openssl verify’ (which is incorrect).
  • Only 9% people adjusted the commands after copy-pasting them from the tutorial.

For explanation and further details on there (and other) numbers, see the paper text.

Publication summary at DevConf 2018

Citation

Why Johnny the Developer Can’t Work with Public Key Certificates: An Experimental Study of OpenSSL Usability. Martin Ukrop and Vashek Matyas, Topics in Cryptology – CT-RSA 2018: The Cryptographers’ Track at the RSA Conference 2018, Springer International Publishing, 2018, 45–64.

Authors

Partner University

Institutes

Associated Research Projects