Assessing Real-World Applicability of Redesigned Developer Documentation for Certificate Validation Errors [EuroUSEC 2022]

   Authors: Martin Ukrop, Michaela Balážová, Pavol Žáčik, Eric Vincent Valčík and Vashek Matyas

 Primary contact: Martin Ukrop <>

 Conference: EuroUSEC 2022

   DOI: 10.1145/3549015.3554296

PDF   Artifacts   Presentation   BiBTeX

  Title         = {Assessing Real-World Applicability of Redesigned Developer Documentation for Certificate Validation Errors},
  Author        = {Martin Ukrop and Michaela Balážová and Pavol Žáčik and Eric Vincent Valčík and Vashek Matyas},
  BookTitle     = {Proceedings of the 2022 European Symposium on Usable Security},
  Series        = {EuroUSEC '22},
  Publisher     = {ACM},
  Location      = {Karlsruhe, Germany},
  Year          = {2022},
  Pages         = {131–144},
  DOI           = {10.1145/3549015.3554296},


We face certificate validation errors commonly, yet the related tools and documentation had been shown to have very poor usability. Previous research suggests that just improving the error messages and corresponding documentation can have significantly positive effects. Our work aims at increasing the usability of certificate validation by 1) redesigning the API error messages and the corresponding documentation, and 2) validating the real-world applicability of the redesign by investigating the opinions of 180 IT professionals. We focus on the perceived obstacles, desired ideal form and overall satisfaction. The redesigned documentation exhibits a reliable significant decrease in perceived incompleteness, with a small amount of perceived bloat and tangle. The redesigned documentation, now published on a dedicated website, is preferred by 89% of our study participants.

The artifacts accompanying this paper contain three major parts:

  1. The questionnaire used in the main study (described in Sections 3.1 and 3.2 of the paper and mostly present in Appendices A and B of the paper).
  2. The anonymized dataset (multiple formats) of all valid questionnaire answers and qualitative coding performed. Analyses of this dataset are the core of the paper and are present in subsection 3.4 and all parts of Sections 4 and 5.
  3. The set of analyses files (IBM SPSS scripts and outputs) producing all statistical results presented in the paper are included.

More details about the artifacts can be found in the README file in the artifacts archive.

Artifacts (ACM Digital Library)