Get your FREE copy of "The Ultimate Guide of SSL"

Download Ebook

A Massive Jolt: Let’s Encrypt Is Revoking 3 Million SSL Certificates

Without Checking CAA Field of the Requesting Domain, Let’s Encrypt Issued 3,048,289 SSL/TLS Certificates

This March 4th, 2020, will be remembered for decades in the history of Let’s Encrypt. Today, at 9:00 p.m. EST, Let’s Encrypt the world’s biggest free SSL/TLS Certificate provider will revoke more than 3 million SSL/TLS Certificate due to the CAA (Certificate Authority Authorization) rechecking bug.

Precisely, the bug has impacted the CAA implementation specification, which is inside Boulder: the server software used by Let’s Encrypt project for verifying users and the domains before the issuance of SSL/TLS certificate. Another significant impact is an announcement that came upon suddenly, which barely gave time to their users.

Ultimately, this move of Let’s Encrypt means, millions of machine identities and websites that rely upon Let’s Encrypt certificate like you or other users for protecting their sensitive data flow will be looked upon as insecure and in some cases, it may even be rendered unavailable, and the worst is users won’t be aware of it.

Many Let’s Encrypt certificate users got the revocation notification on Tuesday, and 24 hours were given for resolving the issue.

We’re a bit late in covering this news, and it’s possible you might have already read it somewhere. But, if you haven’t and you’re reading about it for the first time, then you might be questioning certain things such as why this revocation is taking place? What does it mean for the users of Let’s Encrypt? What to do if you’re among those whose certificate has been affected?

No worries, we’ll cover all of it along with some other important ones. Just go through this article and get your questions answered.

Let’s get into it.

Major Revocation of Let’s Encrypt SSL/TLS Certificates

letsencrypt-error
On Feb 29th, 2020, Let’s Encrypt found out there’s a bug within the code that allows the issuance of an SSL/TLS certificate that hasn’t gone through proper domain record checks. Thus, it resulted in today’s major revocation of the total of 3,048,289 SSL/TLS Certificates out of the total 116 million active Let’s Encrypt SSL/TLS certificate, till date. It means roughly 2.6% are impacted due to this bug. And out of this total 3 million affected certificates, 1 million are duplicate certificates, means issued for the same domain and sub-domains, and the other 2 million are the actual ones, which are impacted because of this bug.

Also, the same has been explained by the Let’s Encrypt engineer through special FAQ page, made for this incident: ” Because of the way this bug operated, the most commonly affected certificates were those that are reissued very frequently, which is why so many affected certificates are duplicates”.

What to Do if Your Let’s Encrypt SSL Certificate Has Been Impacted?

You have no other option except renewing your affected SSL certificate, and unfortunately, the revocation deadline (last date) is today itself, March 4th, 2020. So, if you fail to renew and Let’s Encrypt revokes your certificate, your website visitors will start receiving security warnings, which will scare the users away.

Also, it’s recommended that you check your mail because they’ve notified the domain owners by email who have been impacted by this bug, though all users have not listed a valid contact method. So, it’s obvious many might not receive it.

lets-encrypt-ssl-revoke-email
Also, lead developer for Let’s Encrypt, Jacob Hoffman-Andrews, has posted on Mozilla’s Bugzilla web forum about the same:

On 2020-02-29 UTC, Let’s Encrypt found a bug in our CAA code. Our CA software, Boulder, checks for CAA records at the same time it validates a subscriber’s control of a domain name. Most subscribers issue a certificate immediately after domain control validation, but we consider a validation good for 30 days. That means in some cases we need to check CAA records a second time, just before issuance. Specifically, we have to check CAA within 8 hours prior to issuance (per BRs §3.2.2.8), so any domain name that was validated more than 8 hours ago requires rechecking.

The bug: when a certificate request contained N domain names that needed CAA rechecking, Boulder would pick one domain name and check it N times. What this means in practice is that if a subscriber validated a domain name at time X, and the CAA records for that domain at time X allowed Let’s Encrypt issuance, that subscriber would be able to issue a certificate containing that domain name until X+30 days, even if someone later installed CAA records on that domain name that prohibit issuance by Let’s Encrypt.”

In case you haven’t received any email, then you need to verify CAA (Certificate Authority Authorization) records, to know whether you’ve been impacted or not.

What’s the Importance of CAA Records in the Issuance Process of SSL/TLS Certificate?

CAA records are used for specifying which Certificate Authorities (CAs) can issue the certificate for a domain. In other words, it works as a resource which helps in avoiding the issuance of fake SSL/TLS certificate for any domain name, while helping to strengthen the PKI ecosystem. However, it’s a DNS record of the domain, which must be verified by every issuing CA. Because it easily let CA know whether they have any authority to issue any certificate for a domain or not. For instance,

  • If there’s a CAA record, then the only listed CA(s) are allowed to issue an SSL/TLS certificate for a particular domain.
  • If there’s no CAA record, it means any CA has the right to issue an SSL/TLS certificate for any domain.
And, that’s also a fact that whenever any SSL certificate gets issued by any Certificate Authority, it’s required to go through certain guidelines outlined in the Baseline Requirements (BR) documentation of the CA/Browser Forum. Ultimately, it means that like other CAs whenever to Let’s Encrypt issue any SSL certificate, first, it needs to check the CAA records (8 hours before the issuance) and a failure to do so due to any reason bug or other, leads to revocation.

Here’s Why Revocation of Let’s Encrypt SSL/TLS Certificate Is Different

Most of CA(s) has revoked SSL/TLS certificates. But the situation is a bit different when it comes to Let’s Encrypt.

Let’s know why.

Everyone Are Not Notified They Are Affected

Let’s Encrypt has done its job for notifying its users. They’ve emailed “affected subscribers for whom we have contact information.” Although that’s also a truth that Let’s Encrypt doesn’t have contact information of all their users, as they are a free certificate provider of a domain validated SSL/TLS certificate, which means not all their users will provide correct contact details. So, failure to notify all the users about updating their installed SSL certificate before the revocation takes place is apparent, and there’s no surety that everyone keeps up with the forum news daily.

However, they have also posted the link of the tool on their Let’s Encrypt forum through which you can know whether your SSL certificate has been affected. And, it can even be an indication of the big issue coming further for Let’s Encrypt users, which can prove costly to the affected organization too. And, as per the Ponemon Institute and KeyFactor, it has been estimated that unexpected certificate outages can cost more than $11 million.

Let’s Encrypt Certificate Subscribers Have Limited Support

If mass revocation is occurring, a reasonable amount of time has to be given to the users, along with lots of resources to help and guide the subscribers. But Let’s Encrypt is a free certificate authority (CA), due to which people have to run behind a bunch of digital documents and web forum conversations, which is not helpful. Especially in such a short deadline.

Using Let’s Encrypt SSL/TLS Certificate – Here’s What You Should Do

If you’re one of those who are not using the certificate of CAs like Sectigo, GeoTrust, Thawte, and has been affected by this Let’s Encrypt mass revocation, then no need to worry. Here, we’ll guide you through proper instructions.

Check Your Installed Let’s Encrypt SSL/TLS Certificates

If you’re not aware of whether you’ve been affected or not. Simply check the serial number of your SSL/TLS certificate and verify whether it matches the list of Let’s Encrypts affected certificates. To do so, you can download the list of those affected certificates mentioned in this link of the serial number and look for lines that start with the account IDs.

Or else, you can also go to this online tool, and if you’re using Linux or system such as BSD, simply run the below command into the interface.

openssl s_client -connect example.com:443 -showcerts </dev/null 2>/dev/null | openssl x509 -text -noout | grep -A 1 Serial\ Number | tr -d :

Certificate Renewal

Once you get confirmed that your certificate is affected, then the only choice you’re left is to renew it. In case you were using the SSL/TLS certificate from the commercial CAs, which offers a 1 to 2 year validity period, unlike Let’s Encrypt, which offers only 90 days, then reissue could’ve been possible, but with Let’s Encrypt that’s not the case.

However, clients who’re using ACME for renewing their certificates, they have to refer their documentation related to SSL/TLS certificate renewal process. And, if you’re using Certbot, use below command:

certbot renew --force-renewal

Also, if you’re using cPanel for managing and installing your Let’s Encrypt SSL certificates, then simply go through regular renewal steps and renew it.

letsencrypt-cpanel
Note: On cPanel, we were able to issue a new SSL/TLS certificate.

Summary

It’s not new many CAs have revoked their issued certificates. And, that’s also the fact that it’s not a fun process. But this mass revocation incident of free CA Let’s Encrypt can create more problems due to many reasons, such as lack of contact details, limited support capability, which makes it hard for clients to re-issue certificates quickly for avoiding downtime. Though, Let’s Encrypt has suggested to refer their help document and also post in the Get Help forums, if any help is needed.

Related Articles:

Disclosure: AboutSSL appreciates your continuous support. It helps us tremendously to keep moving in the competitive SSL industry. Here most of the links which direct you to buy any SSL/TLS related service or products earns us a certain percentage of referral commission. Learn More