Will You Trust This TLS Certificate? Perceptions of People Working in IT

Research cooperation of CRoCS, Faculty of Informatics, Masaryk University and Red Hat.

• Authors: Martin Ukrop, Lydia Kraus, Vashek Matyas and Heider Wahsheh
• Conference: Annual Computer Security Applications Conference (ACSAC) 2019
• Paper details: Page with PDF and supplementary materials

Abstract

Flawed TLS certificates are not uncommon on the Internet. While they signal a potential issue, in most cases they have benign causes (e.g., misconfiguration or even deliberate deployment). This adds fuzziness to the decision on whether to trust a connection or not. Little is known about perceptions of flawed certificates by IT professionals, even though their decisions impact high numbers of end users. Moreover, it is unclear how much does the content of error messages and documentation influence these perceptions.

To shed light on these issues, we observed 75 attendees of an industrial IT conference investigating, different certificate validation errors. Furthermore, we focused on the influence of re-worded error messages and redesigned documentation. We find that people working in IT have very nuanced opinions regarding the tested certificate flaws with trust decisions being far from binary. The self-signed and the name constrained certificates seem to be over-trusted (the latter also being poorly understood). We show that even small changes in existing error messages and documentation can positively influence resource use, comprehension, and trust assessment. Our conclusions can be directly used in practice by adopting the re-worded error messages and documentation.

Selected conclusions

• We investigated perceived trust in five certificate cases: hostname mismatch, self-signed, expired, name constrained and a flawless certificate (as a control case).
• When validating certificates, the trust decisions are not binary. Even IT professionals do not completely refuse a certificate just because its validation check fails.
  • In case of expired certificates, the expiry duration plays an important role: Certificates expired yesterday were mostly considered as “looking OK”, while a certificate expired 2 weeks ago “looks suspicious” and the one expired a year ago seems “outright untrustworthy”.
  • The certificate subject plays a role: Flaws were less likely to be tolerated for big, established companies (Microsoft was mentioned as an example).
• We found some certificate cases as over-trusted.
  • 21% of the participants considered the self-signed certificate as “looking OK” or better, with a trust mean comparable to that of an expired certificate. We find this concerning as the self-signed certificate never had any identity assurances.
  • Similarly, 20% of the participants considered the name constrained certificate as “looking OK” or better, with a trust mean again comparable to that of an expired certificate. We find this concerning as the name constraints violation hints at misconfiguration or even malicious activity at the sub-authority level.
• We had half of the participants interact with real OpenSSL error messages and the other half with our re-designed error messages and documentation. Here is the comparison:
  • The self-signed case was considered significantly less trustworthy with our error message (which we consider a success).
  • The name constrained case was also perceived as less trusted and required less time and less online browsing to understand.
  • The other attributes were comparable – thus, we see our documentation in these cases as better than the existing one.
• In the redesigned error messages, we included a link to the documentation. To our surprise, 71% of the participants clicked this link. This suggests a nice opportunity of directing the developers to a usable place recommended by the library designers.
• As a follow-up work, we started gathering X.509 certificate validation errors and documentation from multiple libraries to consolidate the documentation on a single place.

Publication summary at DevConf 2019

Citation

Will You Trust This TLS Certificate? Perceptions of People Working in IT. Martin Ukrop, Lydia Kraus, Vashek Matyas and Heider Wahsheh, 35rd Annual Computer Security Applications Conference (ACSAC 2019), ACM, 2019.