Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
public:papers:acsac2019 [2019-09-20 20:11] xukroppublic:papers:acsac2019 [2023-08-12 21:03] (current) – [Research artifacts (supplementary material)] xukrop
Line 6: Line 6:
 <TEXT size="large"> <TEXT size="large">
  
-\_{{fa>user}}\_\_//Authors:// Martin Ukrop, Lydia Kraus, Vashek Matyas and Heider Wahsheh+\_{{fa>user}}\_\_//Authors:// [[:publications:authors:martin-ukrop|Martin Ukrop]], Lydia Kraus, [[:publications:authors:vashek-matyas|Vashek Matyas]] and Heider Wahsheh
  
 {{fa>user-circle-o}}\_//Primary contact:// Martin Ukrop %%<%%<mukrop@mail.muni.cz>%%>%% {{fa>user-circle-o}}\_//Primary contact:// Martin Ukrop %%<%%<mukrop@mail.muni.cz>%%>%%
  
 {{fa>bullhorn}}\_//Conference:// [[https://www.acsac.org/|ACSAC 2019]] {{fa>bullhorn}}\_//Conference:// [[https://www.acsac.org/|ACSAC 2019]]
 +
 +\_{{fa>id-badge}}\_\_//DOI:// [[https://doi.org/10.1145/3359789.3359800|10.1145/3359789.3359800]]
 </TEXT> </TEXT>
 </col> </col>
Line 17: Line 19:
 <TEXT align="right"> <TEXT align="right">
  
-<button type="warning" icon="fa fa-file-pdf-o">[[https://crocs.fi.muni.cz/_media/publications/pdf/2019-acsac-ukrop.pdf|Pre-print PDF]]</button>+<button type="warning" icon="fa fa-fw fa-file-pdf-o">[[https://crocs.fi.muni.cz/_media/publications/pdf/2019-acsac-ukrop.pdf|Pre-print PDF]]</button>
 \_ \_
-<button type="primary" icon="fa fa-database">[[https://drive.google.com/drive/folders/1zzGZ0h-tBrntBPvW5qek2wRehWnIwrx-?usp=sharing|Artifacts]]</button>+<button type="primary" icon="fa fa-fw fa-database">[[#research_artifacts_supplementary_material|Artifacts]]</button>
 \_ \_
-<popover trigger="focus" title="Not yet available" content="Presentation will be added in December 2019."> +/*<popover trigger="focus" title="Not yet available" content="Presentation will be added in December 2019.">*/ 
-<button icon="fa fa-file-image-o">Presentation</button> +<button icon="fa fa-fw fa-file-image-o">[[https://crocs.fi.muni.cz/_media/publications/pdf/2019-acsac-ukrop_presentation.pdf|Presentation]]</button> 
-</popover>+/*</popover>*/
 \_ \_
-<button collapse="bibtex" icon="fa fa-file-code-o">BiBTeX</button>+<button collapse="bibtex" icon="fa fa-fw fa-file-code-o">BiBTeX</button>
 </TEXT> </TEXT>
 </col> </col>
Line 34: Line 36:
     Title         = {Will You Trust This TLS Certificate? Perceptions of People Working in IT},     Title         = {Will You Trust This TLS Certificate? Perceptions of People Working in IT},
     Author        = {Martin Ukrop and Lydia Kraus and Vashek Matyas and Heider Ahmad Mutleq Wahsheh},     Author        = {Martin Ukrop and Lydia Kraus and Vashek Matyas and Heider Ahmad Mutleq Wahsheh},
-    BookTitle     = {to appear at 35rd Annual Computer Security Applications Conference (ACSAC'2019)},+    BookTitle     = {35rd Annual Computer Security Applications Conference (ACSAC'2019)},
     Year          = {2019},     Year          = {2019},
     Publisher     = {ACM},     Publisher     = {ACM},
-    crocsweb      = {https://crocs.fi.muni.cz/papers/acsac2019}, 
     Keywords      = {usablesec, Red-Hat},     Keywords      = {usablesec, Red-Hat},
 +    DOI           = {10.1145/3359789.3359800},
   }   }
 </collapse> </collapse>
Line 48: Line 50:
 </panel> </panel>
  
 +===== Selected conclusions =====
 +
 +  * We investigated perceived trust in five certificate cases: hostname mismatch, self-signed, expired, name constrained and a flawless certificate (as a control case).
 +  * When validating certificates, the trust decisions are not binary. Even IT professionals do not completely refuse a certificate just because its validation check fails.
 +    * In case of expired certificates, the expiry duration plays an important role: Certificates expired yesterday were mostly considered as __"looking OK"__, while a certificate expired 2 weeks ago __"looks suspicious"__ and the one expired a year ago seems __"outright untrustworthy"__.
 +    * The certificate subject plays a role: Flaws were less likely to be tolerated for big, established companies (Microsoft was mentioned as an example).
 +  * We found some certificate cases as over-trusted.
 +    * 21% of the participants considered the self-signed certificate as __"looking OK"__ or better, with a trust mean comparable to that of an expired certificate. We find this concerning as the self-signed certificate never had any identity assurances.
 +    * Similarly, 20% of the participants considered the name constrained certificate as __"looking OK"__ or better, with a trust mean again comparable to that of an expired certificate. We find this concerning as the name constraints violation hints at misconfiguration or even malicious activity at the sub-authority level.
 +  * We had half of the participants interact with real OpenSSL error messages and the other half with our re-designed error messages and documentation. Here is the comparison:
 +    * The self-signed case was considered significantly less trustworthy with our error message (which we consider a success).
 +    * The name constrained case was also perceived as less trusted and required less time and less online browsing to understand.
 +    * The other attributes were comparable – thus, we see our documentation in these cases as better than the existing one.
 +  * In the redesigned error messages, we included a link to the documentation. To our surprise, 71% of the participants clicked this link. This suggests a nice opportunity of directing the developers to a usable place recommended by the library designers.
 +  * As a follow-up work, we started gathering X.509 certificate validation errors and documentation from multiple libraries to consolidate the documentation on a single place.
 +
 +<button type="primary" icon="fa fa-fw fa-link">[[https://x509errors.org|Visit x509errors.org]]</button>
 +
 +===== Talk at DevConf 2019 =====
 +
 +The content of this research was partially covered at the DevConf 2019 talk that can be seen below. <button icon="fa fa-fw fa-file-image-o">[[https://crocs.fi.muni.cz/_media/publications/pdf/2019-devconf-ukrop_presentation.pdf|Presentation]]</button>
 +
 +{{ youtube>ezs99TiPDhs?900x520 |Martin Ukrop: Understanding TLS certificate validation errors}}
  
 ===== Research artifacts (supplementary material) ===== ===== Research artifacts (supplementary material) =====
Line 57: Line 82:
 The collected data is presented in a single dataset (SPSS format; you can use PSPP as a free alternative). It includes the analysis syntax files to obtain the numerical results presented in the paper. For each participant, the dataset contains: 1) pre-task questionnaire answers, 2) reported trust ratings, 3) sub-task timing, 4) information on whether they browsed the Internet and 5) the interview codes assigned. Note that we do not publish the interview transcripts to preserve participant privacy. The collected data is presented in a single dataset (SPSS format; you can use PSPP as a free alternative). It includes the analysis syntax files to obtain the numerical results presented in the paper. For each participant, the dataset contains: 1) pre-task questionnaire answers, 2) reported trust ratings, 3) sub-task timing, 4) information on whether they browsed the Internet and 5) the interview codes assigned. Note that we do not publish the interview transcripts to preserve participant privacy.
  
-<button type="primary" icon="fa fa-database">[[https://drive.google.com/drive/folders/1zzGZ0h-tBrntBPvW5qek2wRehWnIwrx-?usp=sharing|Go to artifacts repository (gDrive)]]</button>+<button type="primary" icon="fa fa-fw fa-database">[[https://zenodo.org/record/8242000|Go to artifacts repository (Zenodo)]]</button>