Why Johnny the Developer Can't Work with Public Key Certificates [RSA-CT 2018]

   Authors: Martin Ukrop and Vashek Matyas

 Primary contact: Martin Ukrop <mukrop@mail.muni.cz>

 Conference: RSA Cryptographers' Track 2018

   DOI: 10.1007/978-3-319-76953-0_3

@InProceedings{2018-rsa-ukrop,
  Author        = {Martin Ukrop and Vashek Matyas},
  Title         = {Why Johnny the Developer Can't Work with Public Key Certificates: An Experimental Study of OpenSSL Usability},
  BookTitle     = {Topics in Cryptology -- CT-RSA 2018: The Cryptographers' Track at the RSA Conference 2018},
  Year          = {2018},
  Publisher     = {Springer International Publishing},
  Pages         = {45--64},
  DOI           = {10.1007/978-3-319-76953-0_3},
}

Abstract

There have been many studies exposing poor usability of security software for the common end user. However, only a few inspect the usability challenges faced by more knowledgeable users. We conducted an experiment to empirically assess usability of the command line interface of OpenSSL, a well known and widely used cryptographic library. Based on the results, we try to propose specific improvements that would encourage more secure behavior. We observed 87 developers/administrators at two certificate-related tasks in a controlled environment. Furthermore, we collected participant opinions on both the tool interface and available documentation. Based on the overall results, we deem the OpenSSL usability insufficient according to both user opinions and standardized measures. Moreover, the perceived usability seems to be correlated with previous experience and used resources. There was a great disproportion between the participant views of a successful task accomplishment and the reality. A general dissatisfaction with both OpenSSL interface and its manual page was shared among the majority of the participants. As hinted by a participant, OpenSSL gradually “turned into a complicated set of sharp kitchen knives” – it can perform various jobs very well, but laymen risk stabbing themselves in the process. This highlights the necessity of a usable design even for tools targeted at experienced users.

Several interesting bits...

The usability of OpenSSL interface turned out to be… not so great (no real surprise there). But there are some interesting bits and pieces. To name just a few:

For explanation and further details on there (and other) numbers, see the paper text.

Talk at DevConf 2018

Research artifacts (supplementary material)

The supplementary material to this paper contains of:

Artifacts (Zenodo)