Integration of a new install of a Shibboleth v3 IdP into the UK federation to replace a Shibboleth v2 IdP

This page is a guide for deployers of version 2 of the Shibboleth IdP software who wish to migrate to version 3, and do so without loss of service for their users. This page gives an overview of the steps that should be taken, although the details will depend on your current IdP's deployment. The timescale for migration will depend on a number of factors such as: how complex your v2 deployment is, which Service Providers your IdP connects to, and the familiarity that you have with Shibboleth and SAML.

Read not to contradict and confute; nor to believe and take for granted; nor to find talk and discourse; but to weigh and consider. "The Essays or Counsels, Civil and Moral" by Francis Bacon (available through Project Gutenburg)

Assumptions used in this page

It is possible to do an upgrade of a Shibboleth v2 IdP, but it is not safe in general to upgrade a production installation directly. As the Shibboleth documentation says, you must have a development environment in which to test the upgraded software or you will experience extensive downtime and probably end up reverting the upgrade several times while trying to get back to a working system. This approach will also leave you with a hybrid system that has some new style and some old-style configuration files.

This page starts from the assumption that you'll make a new install on a new server, as we feel that understanding the new configuration files as soon as possible is better in the long run. There are other reasons why you would want to do a fresh install. You may want to use a different operating system or use upgraded components for the new IdP. You may want to take the opportunity to review the attribute release policies in your IdP. Or you may want to use the new features of v3, and an upgrade from v2 doesn't enable them by default.

This page assumes:

  • You already have a Shibboleth v2 IdP registered in the UK federation, with entityID https://idp.example.ac.uk/idp/shibboleth, endpoints on the host idp.example.ac.uk and a display name of "University of Life".
  • You intend to install a test IdP on the host new.example.ac.uk with entityID of https://new.example.ac.uk/idp/shibboleth
  • You do not want any loss of service when you migrate in the new IdP
  • You are using the same scope in the new server: example.ac.uk.
  • You do not want your users to lose any personalisations, so after migration you want the new server to use the same entityID as the old server, and to generate the same eduPersonTargetedIDs for every user.
  • You want to retain WAYFless URLs on your library portal, so after migration you want the the new server to be hosted on idp.example.ac.uk.
  • You will install a v3.2.1 Shibboleth IdP or later. Version 3.2 includes a number of new features and makes changes to the configuration files, such as logback.xml. If you installed v3.1 or earlier, and then upgraded, then the configuration files will not be upgraded to the v3.2 format. Note also that v3.2.1 includes an important bugfix in the example attribute-resolver.xml configuration file. For full details, see the Shibboleth v3 IdP Release Notes.
  • You're aware of the reference documentation on the Shibboleth wiki

Timescales to perform the migration

What we have found is that the migration can take an elapsed time of a few weeks, although there are some caveats:

  • Often, the person responsible for the v3 install is not the person who installed the v2 IdP, and the original installer is not contactable. In this case, the person responsible for the upgrade should spend time to understand the new IdP, its configuration files, integration with the UK federation etc. This is time well spent because the deployer gains experience which is useful for maintaining the IdP post-upgrade. If you contract out the upgrade then the time taken can be shorter. However you should ensure that the contractor transfers enough knowledge so that you can then maintain the system. At the very least, you should find out how to upgrade the IdP software within the 3.x series in case there's a security update.
  • much of that time is spent waiting for metadata to propagate, or waiting for email responses, or for testing, so engaging a contractor for a continuous block of time may be inefficient.

The timescale also depends on how complex the existing IdP is. The following questions gauge how complex the v2 setup is, and you should have an idea about the following questions before your start the v3 install:

  • What is the method of authentication? The simplest is authentication using username/password to an organizational directory (LDAP or Active Directory).
  • The number and complexity of the attributes defined in attribute-resolver.xml. Are there any scripted attributes?
  • Whether your users have accounts on SPs that use personalisations based around eduPersonTargetedID. This make the migration of the new IdP more involved.
  • The number and complexity of the attribute release filters defined in attribute-filter.xml
  • How tightly integrated with the rest of your system is the IdP? Monitoring, system administration, user account management...
  • Are there any custom components added to the IdP?
  • Does the IdP interact with SPs outwith the UK federation? You'd find this in the MetadataProvider elements of the v2 relying-party.xml file

Install the Shibboleth v3 IdP

Reference documentation for installation of the Shibboleth v3 IdP software is here. Please also see the UK federation's IdP v3 installation and configuration guide, although at this point I recommend that you just follow the installation parts of the documentation and look at the configuration parts after the new IdP is registered.

Note that the Shibboleth Project have rewritten the IdP from the ground up, and the configuration files have been rationalised. And by that, I mean that there are more of them, but each of them has a specific role. This is different to the v2 IdP where, for example, the relying-party.xml file controlled several conceptually different functions.

Some of the configuration parameters have been moved to "properties files" which are text files with key-value pairs, and other configuration files can include references to the keys which are then expanded to the appropriate value. For example, the ldap.properties file is used to define LDAP configuration parameters for both authorisation of the user and attribute resolution (different to version 2, where you would have had to store LDAP credentials in both login.config and attribute-resolver.xml.)

You'll also find that the metadata that's created during installation contains 3 distinct certificates, compared to the version 2 IdP's configuration with a single certificate. This is expanded on in the Shibboleth Project documentation on Security and Networking.

In what follows, I'll use %{idp.home} as the on-disk location of your Shibboleth installation.

Register the Test IdP

You should follow our procedure for Registering a Shibboleth IdP. Please note that:

  • The test IdP should be registered as hidden from the default discovery service list.
  • The test IdP must have a different display name to your production IdP. Something like "University of Life TEST (Do not use)" should be appropriate as it puts the test IdP immediately after your production IdP in the full discovery list, and hopefully it's clear to users that they shouldn't use this IdP.
  • You should register logos if your current IdP has logos registered. You can use the same URLs on the test IdP as on the production IdP.
  • You should use the same domain in the idp.scope property in idp.properties.

A note about keys and certificates

The Shibboleth v2 IdP had a single keypair for all cryptographic purposes. Vulnerabilities such as the 2014 Heartbleed attack on openSSL showed that it is better to have a distinct keypair for each purpose, and so the Shibboleth v3 IdP generates 3 keypairs at installation: for TLS use in the back-channel, for XML signing, and for XML encryption. Therefore, you will find 3 certificates in the metadata at idp-metadata.xml.

When you first register the v3 IdP, I recommended that you continue with the 3 keypairs that are generated at install time. We will check the fingerprints of each of those certificates with you. When you are ready to migrate the v3 IdP into production (see below), you can simplify the v3 configuration to match that of the v2.

The definitive documentation is at the Shibboleth wiki page on Security and Networking.

Configure the Test IdP

Guidance for configuring the IdP for the UK federation, and other aspects of configuration, is given on our IdP installation and set-up page:

http://www.ukfederation.org.uk/content/Documents/Setup3IdP

Use some kind of version control on the configuration files right from the start. It'll make testing easier, and can be used when the machine has gone into production.

  • Logging Set the logging configuration to DEBUG to assist during testing. Note that v3 of the IdP was rewritten from the ground up so logging is somewhat different: the config file remains logback.xml, but the classes that you use to control the logs are different; and the flows are generally the same as in v2, but the log messages from the new libraries can be quite different.
  • Consent As we've assumed that you've installed the IdP rather than upgraded, Consent is configured. This is a new feature for the v3 IdP. If you want to disable this, see the disabling attribute release consent page.
  • Reloadable services The IdP now makes reloading services easier, so you can work within a short cycle of edit files, reload, test. This is much quicker than a cycle of edit, restart servlet container, test.

Test the IdP

It's important to test the new IdP step-by-step, changing one thing at a time and testing that the effect is as expected. This will also build familiarity with the new configuration files and functionality.

UK federation Test SP

Once your test IdP has been registered, you will be able to use the UK federation Test SP as a tool to test against. Test the authentication first, then the attribute release.

Authentication

  • Use the UK federation Central Discovery Service Session Initiator.
  • You'll need to use the "search over all sites" option on the Central Discovery Service because you registered your IdP as hidden.
  • Take care with the ldap.properties file. If there is a property that is in the original file but commented out, then that's the default value that the IdP uses. If, on the other hand, there is a property that is not commented out in the original file, do not comment it out or the IdP may fail to initialise as expected. For example, commenting out idp.authn.LDAP.trustCertificates = %{idp.home}/credentials/ldap-server.crt won't cause an error when starting the IdP, but you may find "A software error was encountered that prevents normal operation" error during a test login because this property cannot be found.
  • Your IdP will display its default login page. Version 3 of the IdP has a default login page, rendered using Velocity rather than jsp.
  • You'll have successfully authenticated if you are redirected from your IdP to the UK federation Attribute Viewer

Attribute release in SAML 2

Now you've tested the authentication, you can move onto attribute release.

  • Use the SAML 2 Session Initiator
  • Look at the attributes and values that have been released.
  • The UK federation has many SPs that use eduPersonTargetedID to link personalisations, so if your staff and learners access any of these SPs, you'll need to add the configuration to your attribute-resolver.xml file. There is more on this topic at set up Attribute release.

Attribute release in SAML 1

There are still significant number of SPs which use SAML 1 exclusively in the UK federation (we are working to get these SPs to upgrade their capability). This provides an additional complication because in typical SAML 1 operation, the SP has to make a back-channel Attribute Query to port 8443 of your IdP.

  • Use the SAML 1 Session Initiator. Ensure that the attributes are released. If not, check that port 8443 on your IdP is opened; check that the certificate that protects port 8443 is the one that's in %{idp.home}/credentials/idp-backchannel.crt; and look in your IdP logs.

Testing against production SPs

In general, it's not possible to authenticate at your test IdP and access production SPs. However, for SPs:

  • which do not have personalisations tied to eduPersonTargetedID
  • which allow access based solely on eduPersonScopedAffiliation
  • and which list your test IdP in their local discovery service, or use the UK federation Central Discovery Service

Then you should be able to gain access using your test IdP.

Non-federated SPs

You may have non-UK federation sources of metadata configured into your production IdP. Once you have configured your test IdP to consume their metadata, and made any required relying party configuration changes, you can test these in a similar way to federated SPs.

Migration

Prepare the new server

  • Edit entityID in %{idp.home}/conf/idp.properties to match that of the production IdP
  • If your production IdPs' metadata registration includes logos, ensure that these will be retained after migration to the new IdP.
  • Change the browser-facing certificate and key to one suitable for the production hostname @idp.example.ac.uk rather than the test IdP.
  • Transfer the v2 keypair to the v3 machine. It should be used to replace the 3 keypairs in the test IdP registration. Signing and encryption uses are set in the %{idp.home}/conf/idp.properties file, and the backchannel use is set in your web servlet container.
  • You should transfer the salt used in a ComputedIdConnector in attribute-resolver.xml.

You should be able to test with the aacli client (either %{idp.home}/bin/aacli.sh or %{idp.home}/bin/aacli.bat depending on your OS). This command line client will show you the attributes released by your IdP to a given SP for each user.

Test the v3 IdP before migration

You've now got the new IdP configured as it will be in production. Before swapping it in, it's possible to use a handy DNS trick to test that the new IdP is working with a SAML 2 HTTP-POST binding. It's not a full test, but if it works then it's likely that the other profiles which use the back channel will also work.

  • On your desktop machine, bodge the DNS so that the IP address points to the v3 server rather than your v2 sever. You can now use your desktop with the v3 IdP
  • Test with Test SP first, but only using SAML 2
  • Ensure that persistent-id is the same as the existing IdP (this checks the salt & ComputedIdConnector)
  • Don't forget to revert the DNS settings on your desktop after testing

This works for the SAML 2 front-channel profile: the SP sends an authentication request to the IdP via your web browser, and the IdP sends messages to the SP back through your browser. Bodging your desktop DNS means that you choose the IdP that is referred to by the DNS name idp.example.ac.uk.

This bodge can't be used for profiles that use the backchannel because in the second part of these profiles, the SP makes a call to port 8443 of idp.example.ac.uk. The SP doesn't know the bodged DNS name so it makes a call to your production IdP, rather than your new IdP, and the production IdP doesn't know about the first part of the profile flow.

Point DNS at the new server

Set a low TTL on the DNS in case you need to revert.

Test again

If there are any issues, you can revert the DNS back to the v2 server.

Conclusion

You should be done...

Please get in touch with helpdesk team if you want to comment on this documentation or discuss any of the points.