When Certificates Attack: Shadow Certificates

Jarred McGinnis
May 12, 2021

Who doesn’t like public key certificates? Probably people who don’t pick up their dog’s messes and Hitler. Those are the kind of baddies that aren’t able to appreciate the phenomenal utility that public key infrastructure delivers via what is essentially a simple string of characters in a file, and some very subtle and beautiful properties of integers and prime numbers. 

It’s hard to overstate the transformational shift from symmetric cryptography to public key cryptography. It’s mind-blowing that only the intended recipient of a message is able to read that message, not despite, but because half of the key is publicly available. Additionally, the same method, through signing, guarantees the integrity and authenticity of the message’s sender. 

It’s this combination of utility and simplicity that has led to the pervasiveness of the technology in our every day. Ironically, it is the prevalence of certificates that cause one of its biggest problems and pose the greatest risk to a company’s reputation and business. It’s easy to take for granted your certificates. They are after all just a file. Current enterprise ICT practices have led to extremely quick software development cycles utilising a heterogenous ecosystem of tools and platforms distributed across a variety of internal and third party systems. At some point someone, somewhere, is going to forget about that ‘just a file’ and a critical certificate is going to expire. This partly explains why enormous global tech brands, time and time again, have suffered embarrassing outages that affected millions upon millions of customers. In 2020, Microsoft, a company that amongst other things makes certificate monitoring tools, suffered a certificate outage in their Teams product, right about when ‘distributed teams’ were about to be all the rage thanks to global lockdowns due to the pandemic. Every organization must not ask if but what to do when a certificate outage will affect them.

What is a certificate outage?

The most widely known and used Transport Layer Security certificates, that give a unique identifier to a bit of kit like a server or even an instance of a software application, are the foundation of a trustworthy and secure network. These certificates come with expiry dates and do so for a number of very good reasons. For example, it limits the amount of time a compromised key can be misused and ensures certificate holders are updated with evolving security measures. With the increasing number of certificates in use, as well as validity periods that are inarguably trending toward shorter, this is only going to be more of an issue. Like a toddler with scissors and a great idea, at this very moment in your ICT infrastructure, is a certificate that people have forgotten about or ignored and it’s about to cause quite a mess. It’s going to expire, and snip, your organization is going to lose millions of dollars and a lot of sleepless nights are going to be spent hunting down the problem, inevitably looking for more typical hardware/software causes before someone identifies that an expired certificate was the problem. 

Nice try, you say, outages don’t scare me, I’ve got a certificate management system (CMS). Well, gather around kiddies, while you hear about...

Shadow Certificates? in my ICT?

Shadow certificates are more likely than you think. It is as if the nails and screws used to build a house end up being what makes the house fall down. Briefly, shadow certificates refer to digital certificates that have been introduced by employees that the DevOps or SecOps teams are unaware of. This could be during the higgledy-piggledy of development or via a third party tool. The consequence is the same: certificate outages. Imagine if you will, a developer working hard, deadlines looming, delivery expected. They’ve just discovered a big problem. They lean in, set their coffee mug down on SecOps best-practices guidelines and get to work solving the problem. For expedience, they self-issue LetsEncrypt or Digicert certificates and these move into production untracked. Another possibility is third-party code includes untracked or, God have mercy on their soul, hard-coded certificates. It may even be the case that the new certificates were placed in the wrong keystore, or the application wasn’t restarted to trigger a keystore reload. Underlying all these is a lack of full visibility on certificates and their use, and a lack of automations to use this visibility information. The consequence is the same; a shadow certificate that will expire and not be renewed. Johnny, tell them what they’ve won… a very angry phone call in the middle of the night, a lot of time and energy wasted, operational downtime, loss of revenues and compliance penalties!

You Can Only Renew What You Know

Cryptosense Analyzer Platform (CAP) ensures complete visibility and awareness of the cryptographic assets being used across the organization. CAP identifies all the cryptographic objects, including what keys and certificates are being used for applications as well as hardware. This means going beyond just scanning for standard certificate file formats, and reconciling those already enrolled in the CMS. CAP scans encrypted keystores, truststores, bytecode, binaries and container images. The newly discovered certificates can be automatically added to the certificate inventory or raised as a JIRA issue with the application owners. 

CAP can be used to verify deployments and ensure that any new certificates being introduced are compliant with company policies or regulatory requirements. Having an accurate, up-to-date inventory of certificates is the essential first step for protecting your organization from certificate outages. No longer relying on development teams to register each and every certificate will make it easier to bring to light -- see what I did there -- all the certificates making it easier to automate the enrollment, provisioning, renewal, and revocation of certificates. Automating certificate policies enables essential cryptographic agility and eliminates outages.

Since CAP is a complete cryptography management platform, it offers more than just full visibility on certificates. It provides an inventory of persistent keys including where and how they are used. It’s possible to generate compliance reports for internal and external cryptography audits in addition to a vulnerability analysis and post-quantum crypto-preparation or cloud-migration readiness scoring.