SERVICE PROVIDER Heartbleed recovery recommendations

Posted on Friday, 18 April 2014

The following announcement has just been sent out to everyone listed as a technical or administrative contact for one or more entities registered with the UK Access Management Federation for Education and Research. Its purpose is to provide recommendations for post-patch mitigation of the "Heartbleed" vulnerability for SERVICE PROVIDER operators.

This advisory is for the attention of SERVICE PROVIDER operators. You should have received the equivalent advisory for the attention of identity provider operators earlier today.

If you have not seen our initial "Heartbleed" advisory, you can find it here: 

 http://www.ukfederation.org.uk/content/News/2014-04-08-secadv

YOU SHOULD ENSURE that the material below is reviewed by your technical staff as soon as possible, so that you can minimise the impact of this issue on your services.

Web Server Vulnerability Impact for Service Providers

If the web server hosting your application was vulnerable at any time, any information visible to that web server may have been compromised. This includes:

  • All data sent to the server by a user's browser. This includes the URLs used to access content, as well as HTML form submissions and cookies used as session tokens.
  • All data sent to a user's browser by the server.
  • All credentials known to the server. This includes the private key associated with the server's SSL/TLS certificate, but also any other private credentials. For example, credentials used to access back-end services such as databases but known to the web server should also be regarded as compromised.
  • The above applies to all virtual hosts handled by the same web server instance, including any sites not visible to the public internet.
  • If the web server shares memory with your application, information from the application such as back-end database credentials and the results of database queries may also have been compromised.

Even if the web server hosting your application was not vulnerable, it's possible that your deployment's SAML processor may use OpenSSL internally (examples of this may include Windows-based SPs running on IIS). If this is the case, the next section applies.

SAML Processor Vulnerability Impact for Service Providers

If the SAML processor component of your deployment uses OpenSSL to perform outbound connections, it may have been vulnerable to Heartbleed independently of the system's web server. The common uses of outbound SSL/TLS connections are:

  • SAML 1 attribute queries
  • SAML 1 or SAML 2 artifact resolution
  • Metadata acquisition (the UK federation metadata is not served on an https:// endpoint, but some other metadata sources are)

If the SAML processor in your system was vulnerable at any time, any information visible to the processor may have been compromised. This includes the private keys associated with the SAML entity's KeyDescriptor elements in metadata.

Note that if the SAML entity's KeyDescriptors make use of the same RSA keys as the web server's browser-facing certificate, the SAML entity can be impersonated without the SAML processor being vulnerable to Heartbleed.

Assessing Your Risk

The Heartbleed vulnerability was introduced into the OpenSSL source code in late 2012. The window of exposure for a particular system will only begin when a vulnerable version of OpenSSL is introduced into your configuration. For example, if your vulnerability came from an upgrade of Red Hat Enterprise Linux to version 6.5 in late November 2013, that would mark the date from which your window of vulnerability would run.

At this point, it is not known whether the Heartbleed vulnerability was known before the announcement on 7th April 2014. Due to the publicity surrounding the vulnerability, however, you should in any case regard it as highly likely that your systems were attacked during the period between the announcement on 7th April and the time at which you patched your systems.

Because the attack leaves no trace in log files, it is impossible to know whether your systems have been attacked during the vulnerability window. Because of the high potential impact of the vulnerability, we therefore recommend that deployers always treat a potential compromise as an actual compromise.

Recovering From Compromises

If the private key for your browser-facing SSL/TLS certificate has been compromised:

  • Create a new 2048-bit RSA key pair
  • Use it to acquire a new certificate from your selected CA
  • Replace the certificate used by your server
  • Revoke the old certificate so that it can no longer be used to impersonate your service

If the private key for your entity's SAML credential has been compromised, either directly or because it is currently the same as your TLS certificate:

  • Create a new 2048-bit RSA key pair
  • Use it to construct a long-lived, self-signed certificate, see 
 http://www.ukfederation.org.uk/content/Documents/SSTFCertificate
  • Contact the UK federation to arrange for a rollover of your SAML configuration and UK federation metadata

Note: The UK federation recommendation is to use separate keys and corresponding certificates for your web server's TLS certificate and your SAML entity's KeyDescriptor metadata. If you currently use the same certificate for both TLS and SAML credentials, you need to treat the SAML credential as compromised as well. We recommend that in this case your new SAML credential uses a long-lived, self-signed certificate rather than using your new TLS certificate.

We recommend that you invalidate all application sessions to avoid session re-use by an attacker. If this is not already performed as part of the web server restart, you will need to perform this operation manually.

Any other credentials known to a vulnerable web server or SAML processor, such as back-end database credentials, should be replaced.

Further Steps

Please contact the UK federation helpdesk (service at ukfederation.org.uk) if you have any additional questions about this advisory, or if you need help in determining which actions to take to recover from the vulnerability.

-- Ian Young, UK federation Edited by SteveGlover on 18 April 2014, at 01:55 PM