May 2020: SSL Certificate Expirations on 30 May 2020 affecting customer operations

We have been contacted by some of our customers who have experienced outages in their backup jobs or attempts to connect to their Wasabi buckets due to problems with a Sectigo Root certificate expiration on May 30, 2020.

These appear to be related to issues with legacy browsers, older applications or systems that do not have the modern “USERTRust” root and would not trust it and so would look further up the chain to a root it does trust, the AddTrust External CA Root. A more modern browser would have the USERTrust root already installed and trust it without needing to rely on the older AddTrust root.

We have updated the Wasabi server certificates to ensure that we have addressed this on our side, but in some cases, it  will be necessary to be sure that you have updated the certificates on your local systems. 

The Wasabi systems were all updated on Saturday May 30, 2020 by 16:11 UTC.  If you have had any service disruptions or errors that fit this timeline and that continue to occur, you may want to take action to update these certificates on your local system. 

For example, 

• Some Legacy clients that did not receive security updates since before mid-2015

  • Apple Mac OS X 10.11 (El Capitan) or earlier

  • Apple iOS 9 or earlier

  • Google Android 5.0 or earlier

  • Microsoft Windows Vista & 7 if the Update Root Certificates Feature has been disabled since before June 2010

  • Microsoft Windows XP if an Automatic Root Update has not been received since before June 2010

  • Mozilla Firefox 35 or earlier

  • Oracle Java 8u50 or earlier

  • Embedded devices (especially copy machines) that have not installed a firmware update since before mid-2015

• Clients configured to explicitly trust one of the expired Roots and ignore the operating system’s or vendor’s managed truststore

• Client software based on OpenSSL library prior to version 1.1.1

• Some OpenLDAP clients

• Java applications that do not use the default truststore

• Clients using cURL tool

• Applications that are connected to by any of the affected clients via SSL/TLS protocol

If your errors point to an SSL certificate issue, we would recommend replacing the cacert.pem on your system. The cacert.pem is a bundle of CA certificates that you use to verify that the server is really the correct site you're talking to (when it presents its certificate in the SSL handshake). The bundle can be used by tools like curl or wget, as well as other TLS/SSL speaking software. The bundle should contain the certificates for the CAs you trust. This bundle is sometimes referred to as the "CA cert store". We have had customers successfully use the cacert.pem file located here, https://curl.haxx.se/ca/cacert.pem (the site hosting the curl application). Your backup software application vendor may have more details on how to ensure that these certificates have been updated for their use.

If you have updated the SSL certificates on your local system, and continue to see issues, please contact support@wasabi.com for further assistance.