IdPv5 upgrade guide
Last Update: 2024-07-01
Introduction
This page contains our guidance for operators of Shibboleth Identity Providers (IdPs) within the UK federation.
Specifically, it is aimed at operators of an existing IdP running the Shibboleth IdP v4 software, that will need to be upgraded to v5.
This upgrade should be completed before September 1, 2024 when the Shibboleth IdP v4 software will reach end of support with the Shibboleth project.
Whilst the UK federation support desk will provide support for operators upgrading their IdP beyond Autumn 2024, we would advise using the following guide, in addition to documentation on the official Shibboleth IdP v5 Wiki, and upgrading as soon as possible, to ensure that you can receive necessary security updates to the software.
For organisations requiring in-depth and hands-on support we offer a Trust and Identity consultancy service
We have made every effort to provide guides that will work for you; however you are encouraged to read in full the guides before starting work, confirm their validity for your deployment and to make backups of your system before beginning.
Using the guide
Linux-specific
Linux platform support
You should be running a current, supported distribution and release of Linux. It can be quite hard to determine when support will end for a given platform. Check https://endoflife.software/operating-systems/ for your platform and release.
You will need to know the Linux distribution and release you are running. Most distributions provide a standard command:
lsb_release -a
This will provide output similar to:
Distributor ID: Ubuntu Description: Ubuntu 22.04.4 LTS Release: 22.04 Codename: jammy
If that doesn't provide the information then you may find it by running:
cat /etc/os-release
Does my Linux distribution supply a recent enough Java and web container?
This guide covers both the Tomcat and Jetty web containers. Different Linux distributions and releases have different versions of Java and Tomcat or Jetty available.
To run IdP5 You will need Java 17 (or later) and either Tomcat 10.1 (or later) or Jetty 11 (or later) so you may need to upgrade various components.
The tables below will allow you to check if your distribution makes suitable versions of the web containers available.
If you're running a generally supported distribution release which doesn't provide the relevant supporting packages then you will need to either upgrade/migrate to one that does or manage the software dependencies yourself.
If you've visited our pages before you may note that CentOS is missing. Cloud providers (eg AWS) along with Rocky are now down-stream distributions of RedHat.
Tomcat
| Distibution | Release | Java available | Java suitable | Tomcat available | Suitable | End of life |
|---|---|---|---|---|---|---|
| RedHat | 9 | 17 | Yes | 10 (manual) | Yes | beyond 2031 |
| Debian | 11 bullseye | 17 | Yes | 10 (manual) | Yes | LTS June, 2026 |
| Debian | 12 bookworm | 17 | Yes | 10 (package) | Yes | LTS June, 2028 |
| Ubuntu | 22.04 LTS | 17 | Yes | 10 (manual) | Yes | April, 2027 |
| Ubuntu | 24.04* LTS | 17 | Yes | 10 (package) | Yes | April, 2029 |
- Please note that Ubuntu 24.04 was released in April 2024 and you may wish to have a discussion around your risk appetite in adopting such a new distribution
If you already have a suitable version of Java installed, you can skip ahead to Tomcat upgrade for Linux deployers.
Jetty
It can be tricky obtaining a suitable version of Jetty, the following table describes the situation for major distributions.
| Distibution | Release | Java available | Java suitable | Jetty available | Suitable | End of life |
|---|---|---|---|---|---|---|
| RedHat | 9 | 17 | Yes | 11 (manual) | Yes | beyond 2031 |
| Debian | 11 bullseye | 17 | Yes | 11 (manual) | Yes | LTS June, 2026 |
| Debian | 12 bookworm | 17 | Yes | 11 (manual) | Yes | LTS June, 2028 |
| Ubuntu | 22.04 LTS | 17 | Yes | 11 (manual) | Yes | April, 2027 |
If you already have a suitable version of Java installed, you can skip ahead to Jetty upgrade for Linux deployers.
Java upgrade - Linux
In general, you should select a Linux distribution and release which provides a supported Java runtime.
Support matrices for popular combinations of platform, Java and web container are given above.
Using the distributions package manager (yum, dnf, apt etc) is the most convenient way to manage the available Java.
The following Java distributions are fully supported:
- Amazon Corretto 17 for Linux
- Red Hat's OpenJDK 17 for Linux as supplied under Red Hat Enterprise Linux (or compatible rebuild)
Note that this specifically refers to actual Java 17, and not “latest”. Some older versions include “floating” versions but not a specifically supported package for Java 17.
Other Java distributions that are substantially identical or meant to be fully compatible with OpenJDK 17 are of course likely to work, but are officially regarded as unsupported.
First check which version is currently installed:
$ java -version openjdk version "11.0.22" 2024-01-16 OpenJDK Runtime Environment (build 11.0.22+7-post-Ubuntu-0ubuntu222.04.1) OpenJDK 64-Bit Server VM (build 11.0.22+7-post-Ubuntu-0ubuntu222.04.1, mixed mode, sharing)
If you need to upgrade then find the appropriate package name using the package manager's search function:
- CentOS / RedHat:
yum search openjdk - Debian / Ubuntu:
apt-cache search openjdk
When you've found the package, install it using the package manager. Below are some examples but check first:
# CentOS / RedHat yum install java-17-openjdk # Debian / Ubuntu apt-get install openjdk-17-jdk
Verify the new version is being used by default:
$ java -version openjdk version "17.0.10" 2024-01-16 OpenJDK Runtime Environment (build 17.0.10+7-Ubuntu-122.04.1) OpenJDK 64-Bit Server VM (build 17.0.10+7-Ubuntu-122.04.1, mixed mode, sharing)
If the install process has not enabled by default then you will need to use the update-alternatives system to select it:
update-alternatives --config java
Remember to check your JAVA_HOME is set correctly after upgrading.
Note: Java 17 does not include a JavaScript Engine like it did in earlier versions, you will need a plugin like Nashorn, there are instructions how to do this in the Shibboleth IdP v4.3.3 -> v5.1.2 upgrade guide on this page.
Tomcat upgrade for Linux deployers
Tomcat upgrade/install
There are incompatibilities between Shibboleth v4.3.1 and Tomcat v10, so if you start this process you are strongly urged to snapshot your working VM before attempting the upgrade (or better, if you have a development or staging environment, perform the update there first)
If your distribution has the required version of Tomcat available then your first choice should be that package. Please see your distribution's documentation on how to install or upgrade packages.
However, should you have Tomcat installed from your distribution's package manager and this version is not appropriate for Shibboleth IdP v5, you will need to manually install the latest version (you can of course run the two versions side by side but you will need to put the configuration and installation and logs in their own directories in order to avoid clashes)
Manual installation of Tomcat
Should your distribution not provide an appropriate Tomcat package then you will need to install and maintain this yourself. The procedure below is just one way of doing this and may need altering to suit your environment.
Note
- Before doing this, ensure you have a supported version of Java available.
Overview
We're assuming you are upgrading from v4 and Tomcat9
We will be using Tomcat10 as both webserver and servlet container, removing the need for Apache to act as a proxy.
- Download the latest version from Tomcat 10 and verify
- Decompress the archive
- Create a tomcat user if required
- Prepare the new directory tree (move bundled directories and symlink to system locations)
- Create master symlink
- Create service
- Configure Tomcat
- Disable (or uninstall) Tomcat9 (if it's already installed)
- Start Tomcat service
Details
Download the latest version
Check the latest version of Tomcat 10 available. Download the package and verify the signature to ensure it's not been tampered with:
For example:
$ cd /usr/local/src $ curl -O https://dlcdn.apache.org/tomcat/tomcat-10/v10.1.19/bin/apache-tomcat-10.1.19.tar.gz $ curl -O https://downloads.apache.org/tomcat/tomcat-10/v10.1.19/bin/apache-tomcat-10.1.19.tar.gz.sha512 $ sha512sum -c apache-tomcat-10.1.19.tar.gz.sha512 apache-tomcat-10.1.19.tar.gz: OK
Decompress the archive
cd /opt tar xfz /usr/local/src/apache-tomcat-10.1.19.tar.gz
If required, create a tomcat user
Check if a tomcat user is already available:
getent passwd tomcat
If nothing is returned then you will need to create a user to run tomcat as. This can vary between distributions but generally looks like:
useradd --system --no-create-home --home-dir /opt/tomcat10 tomcat
Prepare the directory tree
To make upgrades easier, move the components that need to persist across Tomcat upgrades out to their traditional system locations and replace them with symlinks. Lastly, ensure the required directories are writeable by the tomcat user.
mkdir -p /var/cache/tomcat10 /var/lib/tomcat10 /var/log/tomcat10 /etc/tomcat10
cd /opt/apache-tomcat-10.{rest of directory name}
mv -i conf/* /etc/tomcat10/
mv -i logs /var/log/tomcat10/
mv -i temp /var/cache/tomcat10/
mv -i webapps /var/lib/tomcat10/
mv -i work /var/cache/tomcat10/
ln -s /etc/tomcat10 conf
ln -s /var/log/tomcat10 logs
ln -s /var/cache/tomcat10/temp temp
ln -s /var/lib/tomcat10/webapps webapps
ln -s /var/cache/tomcat10/work work
chown -R tomcat /var/log/tomcat10 /var/cache/tomcat10 /var/lib/tomcat10 /etc/tomcat10
chown -R tomcat /opt/apache-tomcat-10.{rest of directory name}
- note that if you have an existing tomcat user on your system that is not "tomcat" - replace with that user in the above chown command.
Create the master symlink
To make upgrades easier, create a symlink to the current version:
cd /opt
ln -s apache-tomcat-10.{rest of directory name} tomcat10
Create the Tomcat service
Most modern Linux distributions use systemd to manage services. Some distributions use "RC" scripts instead.
To see if your system uses systemd, run:
systemctl status
If this returns information about running services then you can add a new service by creating a new service unit file.
nano /etc/systemd/system/tomcat10.service
Make the contents of the file:
[Unit] Description=Tomcat10 After=syslog.target network.target [Service] Type=forking Environment="JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64" Environment="JAVA_OPTS=-Djava.security.egd=file:///dev/urandom" Environment="CATALINA_BASE=/opt/tomcat10" Environment="CATALINA_HOME=/opt/tomcat10" Environment="CATALINA_PID=/opt/tomcat10/temp/tomcat.pid" Environment="CATALINA_OPTS=-Xms512M -Xmx1024M -server -XX:+UseParallelGC" ExecStart=/opt/tomcat10/bin/startup.sh ExecStop=/opt/tomcat10/bin/shutdown.sh RestartSec=10 Restart=always User=tomcat AmbientCapabilities=CAP_NET_BIND_SERVICE ReadWritePaths=/opt/shibboleth-idp [Install] WantedBy=multi-user.target
- NOTE that
JAVA_HOMEin the above can be found by running:ls -l /etc/alternatives/javaand taking the path=]. - NOTE that again, if a preexisting tomcat user is different to "tomcat", replace in the above text.
- NOTE the line above AmbientCapabilities=CAP_NET_BIND_SERVICE which tells systemd to allow Tomcat10 to access privileged network ports.
Save and quit (Ctrl-X and follow the prompts).
Next, stop Apache HTTP Server:
apachectl stop
Next, tell systemd to enable your Tomcat v10 installation. You may also want to disable any existing Tomcat v9 installation:
systemctl enable tomcat10 systemctl disable tomcat9
You may need to uninstall Tomcat9
On Ubuntu (and possibly other distributions) this is important as it removes Tomcat/Apache library(-ies) which Tomcat10 may complain are out of date. Do this if disabling doesn't work in the first instance.
Lastly tell systemd to reload its configuration:
systemctl daemon-reload
Configure Tomcat
The Tomcat configuration requires a few tweaks to support the IdP.
Note that we're assuming an installation of Tomcat v10 (note the directory names in the following). You will need to alter the directory names if you've chosen a different naming convention for any reason.
Note The following moves away from using Apache as a proxy for web requests on port 443 being passed into Tomcat and onto the IdP. The following uses Tomcat as the Web server and Servlet container. This means that Apache will not be able to run alongside.
idp.xml
mkdir -p /etc/tomcat10/Catalina/localhost nano /etc/tomcat10/Catalina/localhost/idp.xml
Make the contents of this file:
<Context docBase="/opt/shibboleth-idp/war/idp.war"
privileged="true"
antiResourceLocking="false"
swallowOutput="true">
<!-- Work around lack of Max-Age support in IE/Edge for Tomcat 8.0.x -->
<CookieProcessor alwaysAddExpires="true" />
<Parameter name="idp.home" value="/opt/shibboleth-idp" override="true" />
</Context>
catalina.properties
Add the following to the end of this file:
# HTTP connector # Allows use of default IdP command line tools. # tomcat.http.host=127.0.0.1 # tomcat.http.port=80 # The HTTPS connector # The interface to listen on. To listen on all interfaces, set tomcat.host = 0.0.0.0 tomcat.https.host=0.0.0.0 tomcat.https.port=443 # pem file variables tomcat.https.pem.certificateFile=<ssl-cert-directory>/<idp certificate file.crt> tomcat.https.pem.certificateKeyFile=<ssl-cert-directory>/<idp certificate's key file.key> tomcat.https.pem.certificateChainFile=<ssl-cert-directory>/<chain file.pem>
- TLS certificateYou will note above that we specify the web-facing TLS certificate in this part of the Tomcat configuration (since we're not using Apache). Modern versions of Tomcat (including 10) allow use of PEM files just as one uses with Apache. When expanding the variables above you must make sure that the tomcat user can read the three files.
server.xml
You need to add two new connectors (one for port 80 and one for port 443).
Paste the following within the <Service name="Catalina"> block, at its start:
<!-- Allows use of default IdP command line tools. -->
<Connector address="${tomcat.https.host}"
port="${tomcat.https.port}"
protocol="org.apache.coyote.http11.Http11NioProtocol"
SSLEnabled="true"
scheme="https"
secure="true">
<SSLHostConfig ciphers="ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-CHACHA20-POLY1305"
disableSessionTickets="true"
honorCipherOrder="false"
protocols="TLSv1.2, TLSv1.3">
<Certificate certificateFile="${tomcat.https.pem.certificateFile}"
certificateKeyFile="${tomcat.https.pem.certificateKeyFile}"
certificateChainFile="${tomcat.https.pem.certificateChainFile}" />
</SSLHostConfig>
</Connector>
Connector blocks.
Setting the default page for contexts outside /idp/ along with http error trapping
cd /opt/tomcat10/webapps cp -r ROOT ROOT-vanilla cd ROOT rm * # note this will deliberately not remove the WEB-INF directory that's in /opt/tomcat10/webapps/ROOT
Create an index.html that looks like the page at /idp/
Next, edit the web.xml file in WEB-INF:
vim WEB-INF/web.xml
Remove the following text:
<display-name>Welcome to Tomcat</display-name>
<description>
Welcome to Tomcat
</description>
Add the following before the final closing tag in the file:
<!-- Other HTTP errors handled here -->
<error-page>
<error-code>404</error-code>
<location>/index.html</location>
</error-page>
<error-page>
<error-code>403</error-code>
<location>/index.html</location>
</error-page>
<error-page>
<error-code>500</error-code>
<location>/index.html</location>
</error-page>
Start the Tomcat service
systemctl start tomcat10
Check the startup logs:
journalctl -fcat /var/log/tomcat10/catalina.out
Tomcat Troubleshooting
If Tomcat cannot deploy the warfile with the set docBase in the idp.xml file in /etc/tomcat10/Catalina/localhost/idp.xml you can try copying the idp.war file into the /opt/tomcat10/webapps directory to see if it will load, this will let you know if there is a permissions issue accessing the warfile directly directly from the docBase path.
Check to see if the idp.war loads and is referenced in /opt/tomcat10/logs/catalina.out like below.
INFO [main] org.apache.catalina.startup.HostConfig.deployWAR Deploying web application archive [/var/lib/tomcat10/webapps/idp.war]
Note: /var/lib/tomcat10/webapps was symlinked earlier from /opt/tomcat10/webapps
Upgrade of manually installed Tomcat - beyond initial v9 -> v10
Once you're managing Tomcat as described above, the upgrade process is:
- Download the latest version (check Shibboleth IdP requirements) from Tomcat and verify
- Decompress the archive
- Prepare the new directory tree (remove bundled directories and symlink system locations)
- Replace master symlink
- Restart Tomcat
This may look like:
# Download the latest version
cd /usr/local/src
curl -O {tar.gz file}
curl -O {tar.gz.sha512 file}
sha512sum -c {tar.gz.sha512 file}
# Decompress the archive
cd /opt
tar xfz /usr/local/src/{tar.gz file}
# Prepare the new directory tree
cd {directory that you uncompress the tar.gz file into}
mkdir ~/tomcat10-backup-1970-01-01
mv -fr conf logs temp webapps work ~/tomcat10-backup--1970-01-01
ln -s /etc/tomcat10 conf
ln -s /var/log/tomcat10 logs
ln -s /var/cache/tomcat10/temp temp
ln -s /var/lib/tomcat10/webapps webapps
ln -s /var/cache/tomcat10/work work
chown -R tomcat .
# Replace master symlink
cd /opt
ls -l tomcat10 # note current version
lrwxrwxrwx. 1 root root YY MMM DD HH:MM tomcat10 -> apache-tomcat-10.something
rm -f tomcat10
ln -s {directory that you uncompressed the tar.gz file into} tomcat10
# Restart Tomcat
systemctl restart tomcat10
Jetty upgrade for Linux deployers
Shibboleth IdP v5 requires Jetty v11 or greater. The Shibboleth project has released an IdP Jetty Configuration Baseline (jetty-base) supporting Jetty v12, and our advice to deployers of Jetty with IdP v5 on Linux is to upgrade to Jetty v12 to ensure current security updates and IdP interoperability.
The recommended container implementation by the Shibboleth project is Jetty, and all development and most testing time by the core project team is confined to the Jetty platform. At present, Jetty v11 or v12 can be used, however, as of January 2024 Jetty v11 has already reached 'end of community support' although security updates will still be provided for now until it's deemed end of life, which we expect will be in January 2025, going on previous lifespans.
Current and up-to-date Jetty packages, such as Jetty v12 tend not to be available for the majority of distributions directly from their main repositories for numerous reasons, because of this a manual installation of Jetty 12 will be required.
You can contact the UK federation service desk at any time for advice with any Jetty upgrade queries.
Note: Back-channel support is no longer supported in Jetty 12!
Manual installation of Jetty 12
Note: Before attempting the Jetty upgrade, ensure you have a supported version of Java available.
Overview
- Back up your current system!
- Download the latest version of Jetty 12 and verify
- Decompress the archive for Jetty
- Download the IdP jetty-base files
- Decompress the archive for jetty-idp-base
- Configure Jetty Modules and JVM Settings (start.ini)
- Configure HTTP/HTTPS Connectors/Certificates
- Configure IdP Context Descriptors (idp.xml)
- Configure Jetty Logging
- Add the Jetty Service (if this is a fresh IdP install)
Back up your current system
Back up your system either by taking a snapshot of your virtual machine, or by creating tar archives, or a combination of these.
Download the latest version of Jetty 12
Ensure you download the latest Jetty 12 version to ensure security and interoperability with IdP v5: https://jetty.org/download.html
Change to the /usr/src dir and download Jetty 12:
Note: The URLs below may not reflect the current version of Jetty, always ensure you are downloading the latest version!
cd /usr/src curl -O https://repo1.maven.org/maven2/org/eclipse/jetty/jetty-home/12.0.10/jetty-home-12.0.10.tar.gz curl -O https://repo1.maven.org/maven2/org/eclipse/jetty/jetty-home/12.0.10/jetty-home-12.0.10.tar.gz.sha1
Check the message digest match:
sha1sum jetty-home-12.0.10.tar.gz cat jetty-home-12.0.10.tar.gz.sha1
Decompress the archive for Jetty 12
Untar the Jetty files:
tar xf jetty-home-12.0.10.tar.gz
Move the dir to where you want it e.g. /opt:
mv jetty-home-12.0.10 /opt/jetty-home
Download the IdP Jetty Base using git
Get the latest IdP jetty-base files for the Shibboleth IdP using git, checkout branch 12 and copy to /opt/jetty-base, then switch to that directory:
cd /usr/src git clone https://git.shibboleth.net/git/java-idp-jetty-base cd java-idp-jetty-base git checkout 12 cp -r src/main/resources/jetty-base /opt/jetty-base cd /opt/jetty-base
Set your JETTY_HOME and JETTY_BASE:
export JETTY_HOME=/opt/jetty-home export JETTY_BASE=/opt/jetty-base
We strongly recommend placing all IdP-specific Jetty configuration under JETTY_BASE to facilitate Jetty upgrades. Do not place that directory inside the IdP installation directory; they can be siblings as desired. Note that Jetty 12 requires the recommended home/base split; the distribution itself is always considered read-only.
Configure Jetty Modules and JVM Settings
The bulk of the configuration is established by setting properties in "ini" files that are combined in the start.d directory. Some of the properties are defined by Jetty and configure built-in modules and others are specific to the IdP and configure the custom module we created.
Add the following to a new file in the start.d directory or check the following matches the current start.ini file: /opt/jetty-base/start.d/start.ini:
# Any other required Jetty modules... # Allows setting Java system properties (-Dname=value) # and JVM flags (-X, -XX) in this file # NOTE: spawns child Java process --exec # Uncomment if IdP is installed somewhere other than /opt/shibboleth-idp #-Didp.home=/path/to/shibboleth-idp # Maximum amount of memory that Jetty may use, at least 1.5G is recommended # for handling larger (> 25M) metadata files but you will need to test on # your particular metadata configuration. If MDQ metadata on demand is used, # requirements may be much lower. -Xmx1500m # Prevent blocking for entropy. -Djava.security.egd=file:/dev/urandom # Set Java tmp location -Djava.io.tmpdir=tmp
The start.d/idp.ini file that configures the “idp” module contains not only the bulk of the basic settings needed but is also a place you can add your own settings to control JVM startup, change which logging module is used, etc. Or you can continue to use a separate start.ini file as above.
Configure HTTP/HTTPS Connectors/Certificates
- Open /opt/jetty-base/start.d/idp.ini
Copy over your cert from the old jetty credentials location directory to /opt/jetty-base/credentials/ and add passwords and correct location to your p12 cert in the idp.ini file. The remaining default settings should be correct for most installs.
The basic HTTP/HTTPS port, address, etc. configuration can be handled within the custom "idp" module and the /opt/jetty-base/start.d/idp.ini property file. The idp.ini example below shows some of the basic properties you can use to configure networking and TLS credentials.
Note: Back-channel support is no longer supported in Jetty 12!
An example idp.ini file for reference:
# --------------------------------------- # Module: idp # Shibboleth IdP # --------------------------------------- --module=idp ## Keystore file path (relative to $jetty.base) jetty.sslContext.keyStorePath=../credentials/idp-userfacing.p12 ## Truststore file path (relative to $jetty.base) jetty.sslContext.trustStorePath=../credentials/idp-userfacing.p12 ## Keystore type jetty.sslContext.keyStoreType=PKCS12 ## Truststore type and provider jetty.sslContext.trustStoreType=PKCS12 ## Keystore password jetty.sslContext.keyStorePassword=changeit ## Truststore password jetty.sslContext.trustStorePassword=changeit ## KeyManager password jetty.sslContext.keyManagerPassword=changeit ## Deny SSL renegotiation jetty.sslContext.renegotiationAllowed=false ## Connector host/address to bind to # jetty.ssl.host=0.0.0.0 ## Connector port to listen on jetty.ssl.port=443 # Allows use of default IdP command line tools. jetty.http.host=127.0.0.1 jetty.http.port=8080
The TLS credential example relies on a PKCS12? file containing the X.509 certificate and private key used to secure the HTTPS channel that users access during authentication and other browser-based message exchanges involving the IdP. This is generally the one you get from a browser-compatible CA, and the example shows it being loaded from a directory inside the JETTY_BASE tree. 8080 is the http port for Jetty (jetty.http.port=8080), It's also okay to leave this unset. That change will confine Jetty to listen locally only rather than over the network.
Configure IdP Context Descriptor
In order to deploy the IdP war file, Jetty must be informed of the location of the IdP warfile. You can of course change the location of this warfile to match your IdP install path.
You need to be careful here as the context descriptor class specified will be specific to the version of Jetty you're running, so do not copy over old jetty-base idp.xml files (this is not recommended to do at all as these files will change over time), you may want to adjust the path to point to your idp.war file, the default is normally /opt/shibboleth-idp/war/idp.war, check the context path where the war file will be launched from matches what's in this idp.xml file. Check that permissions are set correctly for the path so Jetty can get to it. If the idp.war file does not appear to be running for you, try copying the war file to /opt/jetty-base/webapps, if it runs from there then there are permission issues to the default path specified in the idp.xml file.
Open /opt/jetty-base/webapps/idp.xml and check the path to your warfile is correct.
<?xml version="1.0"?> <!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd"> %comment% =============================================================== %% %comment% Configure the Shibboleth IdP webapp %% %comment% =============================================================== %% <Configure class="org.eclipse.jetty.ee9.webapp.WebAppContext"> <Set name="war"><SystemProperty name="idp.home" default="/opt/shibboleth-idp" />/war/idp.war</Set> <Set name="contextPath"><SystemProperty name="idp.context.path" default="/idp" /></Set> <Set name="extractWAR">false</Set> <Set name="copyWebDir">false</Set> <Set name="copyWebInf">true</Set> </Configure>
Jetty Logging
The recommended approach is to use logback for all Jetty logging, the logging module should have already been added to idp.ini, however, you will need to run a command to download the appropriate files from the Maven repo to ensure you have the correct libraries for logging to work.
Download dependency files for logging:
cd /opt/jetty-base
java -jar /opt/jetty-home/start.jar --add-module=logging-logback
ALERT: There are enabled module(s) with licenses.
The following 1 module(s):
+ contains software not provided by the Eclipse Foundation!
+ contains software not covered by the Eclipse Public License!
+ has not been audited for compliance with its license
Module: logging-logback
+ Logback: the reliable, generic, fast and flexible logging framework.
+ Copyright (C) 1999-2012, QOS.ch. All rights reserved.
+ This program and the accompanying materials are dual-licensed under
+ either:
+ the terms of the Eclipse Public License v1.0
+ as published by the Eclipse Foundation:
+ http://www.eclipse.org/legal/epl-v10.html
+ or (per the licensee's choosing) under
+ the terms of the GNU Lesser General Public License version 2.1
+ as published by the Free Software Foundation:
+ http://www.gnu.org/licenses/old-licenses/lgpl-2.1.html
Proceed (y/N)? Y
INFO : mkdir ${jetty.base}/start.d
INFO : logging-logback initialized in ${jetty.base}/start.d/logging-logback.ini
INFO : resources transitively enabled
INFO : logging/slf4j dynamic dependency of logging-logback
INFO : mkdir ${jetty.base}/resources
INFO : copy ${jetty.home}/modules/logging/logback/resources/logback.xml to ${jetty.base}/resources/logback.xml
INFO : mkdir ${jetty.base}/lib/logging
INFO : download https://repo1.maven.org/maven2/ch/qos/logback/logback-classic/1.5.6/logback-classic-1.5.6.jar to ${jetty.base}/lib/logging/logback-classic-1.5.6.jar
INFO : download https://repo1.maven.org/maven2/ch/qos/logback/logback-core/1.5.6/logback-core-1.5.6.jar to ${jetty.base}/lib/logging/logback-core-1.5.6.jar
INFO : Base directory was modified
Jetty will populate the /opt/jetty-base/lib/logging dir with the two logback libraries required. It's strongly advisable to check the signatures of the files you downloaded, as below!
curl -O https://repo1.maven.org/maven2/ch/qos/logback/logback-core/1.5.6/logback-core-1.5.6.jar.sha1 sha1sum /opt/jetty-base/lib/logging/logback-core-1.5.6.jar cat /opt/jetty-base/lib/logging/logback-core-1.5.6.jar.sha1 sha1sum /opt/jetty-base/lib/logging/logback-classic-1.5.6.jar curl -O https://repo1.maven.org/maven2/ch/qos/logback/logback-classic/1.5.6/logback-classic-1.5.6.jar.sha1 sha1sum /opt/jetty-base/lib/logging/logback-classic-1.5.6.jar cat /opt/jetty-base/lib/logging/logback-classic-1.5.6.jar.sha1
Open /opt/jetty-base/start.d/idp.ini
Add this to the head of your idp.ini, if it's not already there:
--module=logging-logback
A sample logback configuration for both debug and request logging is provided in /opt/jetty-base/resources/logback.xml if you need to amend anything related to Jetty logging.
Disable Jetty directory browsing
Jetty has had vulnerabilities in the past related to directory browsing and it's recommended to disable directory browsing on Jetty to mitigate any future vulnerabilities around this, as recommended here. There are a few different ways this can be achieved, but one method that's fairly self-contained within the IdP footprint is to modify web.xml.
You can place the below markup at the top of the file under '<display-name>' markup in the web.xml file. e.g. copy the original version from /opt/shibboleth-idp/dist/webapp/WEB-INF/web.xml to /opt/shibboleth-idp/edit-webapp/WEB-INF/web.xml) after modifying the web.xml file, rebuild the war file so this is encompassed into the generated idp.war output file.
Note: the servlet class may differ depending what Jetty version you're running, but should work fine using Jakarta EE 9 servlet in Jetty v12 as below.
<servlet>
<servlet-name>default</servlet-name>
<servlet-class>org.eclipse.jetty.ee9.servlet.DefaultServlet</servlet-class>
<init-param>
<param-name>dirAllowed</param-name>
<param-value>false</param-value>
</init-param>
<load-on-startup>0</load-on-startup>
</servlet>
Rebuild the war file:
cd /opt/shibboleth-idp/bin/ ./build.sh
Add the Jetty Service
The following is a basic example of a systemd service file for starting and stopping the jetty service. You'll need to ensure it's enabled at start-up and that any previous versions of Jetty don't start. If you don't already have a jetty.service file at /etc/systemd/system/jetty.service you can create one as below, note that the Start command below includes the logback configuration File path.
Use POSIX Ambient capabilities to allow use of privileged ports by an unprivileged process, e.g. when using systemd by setting AmbientCapabilities?=CAP_NET_BIND_SERVICE as can be seen in the jetty.service file below.
Note, you will also need to a non-root user account to include in the service file.
[Unit] Description=Jetty provides a Web server and javax.servlet container, plus support for HTTP/2, WebSocket, OSGi, JMX, JNDI, JAAS and many other integrations. After=network.target [Service] WorkingDirectory=/opt/jetty-base ExecStart=/usr/bin/java -Djdk.tls.ephemeralDHKeySize=2048 -Dlogback.configurationFile=resources/logback.xml -jar /opt/jetty-home/start.jar jetty.base=/opt/jetty-base lib=/opt/jetty-home/lib User=jetty AmbientCapabilities=CAP_NET_BIND_SERVICE [Install] WantedBy=multi-user.target
Set the minimum amount of directory/file permissions required for Jetty and always ensure you are not running Jetty as root!
To start the service:
systemctl start jetty
To enable the service at boot:
systemctl enable jetty
Jetty Troubleshooting
Nothing appears to load and keep getting 503 errors restarting attribute-resolver service
Check to see if the warfile path is set correctly in the idp.xml context descriptor file, the path will be broken up into the default path and warfile directory path like below:
<Set name="war"><SystemProperty name="idp.home" default="/opt/shibboleth-idp" />/war/idp.war</Set>
- Check your permissions are set correctly.
- Check the log at: /opt/jetty-base/logs/jetty.log to see if there are any clues as to why Jetty is not starting if not related to the warfile path.
Parent for temp dir not configured correctly
The Java -Djava.io.tmpdir=tmp temp directory in start.ini setting may or may not work depending on your Java setup (ensure your JAVA_HOME is set correctly), you may see an error in jetty.log similar to below, you can get around this by setting an absolute path to the temp directory:
java.lang.IllegalStateException: Parent for temp dir not configured correctly: writeable=false
Linux - Upgrading Shibboleth
Linux - Upgrading Shibboleth IdP from v4.3.1 to v4.3.3
Before you can upgrade Shibboleth IdP to v5.1.2 you must upgrade to v4.3.3, this update was released in March 2024 and must be installed prior to proceeding to the main v5.1.2 update.
After the upgrade to v4.3.3 you will be required to resolve deprecation warnings. Note that some of these warnings may only be visible in your servlet container’s log. This is due to the order of operations; the IdP’s logging service cannot be started until the global Spring context is running, and some of the configuration triggering warnings may be in that root context.
There is one set of warnings that are an exception to this: the undocumented “SAML2NameID” and “SAML1NameID” attribute encoders produce warnings indicating they were removed from the next major version.
DEPRECATED xsi:type ''''SAML2NameID'''', (file [/opt/shibboleth-idp/conf/attribute-resolver.xml]): This will be removed in the next major version of this software; replacement is (none)
In fact they were not, and their use is still valid and there are no explicit plans to remove them.
This upgrade assumes Ubuntu 22.04 LTS and Tomcat10 are installed.
Back up your current system either by taking a snapshot of your virtual machine, or creating tar archives, or a combination of these.
Check the currently installed version
Do not proceed if it less than v4.3.1:
cd /opt/shibboleth-idp ./bin/version.sh 4.3.1
Download the files
cd /usr/local/src curl -O https://shibboleth.net/downloads/identity-provider/latest4/shibboleth-identity-provider-4.3.3.tar.gz curl -O https://shibboleth.net/downloads/identity-provider/latest4/shibboleth-identity-provider-4.3.3.tar.gz.sha256
Check SHA256 message digest matches:
sha256sum -c shibboleth-identity-provider-4.3.3.tar.gz.sha256
Decompress the files
Untar the files and change to the directory:
tar xf shibboleth-identity-provider-4.3.3.tar.gz cd shibboleth-identity-provider-4.3.3
Run the installer for v4.3.3
It should detect your current source/install dir and display that on screen as below:
./bin/install.sh Source (Distribution) Directory (press <enter> to accept default): [/usr/local/src/shibboleth-identity-provider-4.3.3] Installation Directory: [/opt/shibboleth-idp] Update from version 4.3.1 to version 4.3.3 Rebuilding /opt/shibboleth-idp/war/idp.war, Version 4.3.3 Initial populate from /opt/shibboleth-idp/dist/webapp to /opt/shibboleth-idp/webpapp.tmp Overlay from /opt/shibboleth-idp/dist/plugin-webapp to /opt/shibboleth-idp/webpapp.tmp Overlay from /opt/shibboleth-idp/edit-webapp to /opt/shibboleth-idp/webpapp.tmp Creating war file /opt/shibboleth-idp/war/idp.war ...done
Check the version has been upgraded
cd /opt/shibboleth-idp ./bin/version.sh 4.3.3
Although this is a minor update, please check that everything still works before proceeding to the v5.1.2 upgrade.
Reload Attribute Resolver service
This is so there are no issues being flagged on-screen or in the log files
./bin/reload-service.sh -id shibboleth.AttributeResolverService
If reloaded correctly you should see:
Configuration reloaded for 'shibboleth.AttributeResolverService'
Reload Attribute Filter service
Check what attributes are being released and that there are no issues being flagged on-screen or in the log files:
./bin/reload-service.sh -id shibboleth.AttributeFilterService
If reloaded correctly you should see:
Configuration reloaded for 'shibboleth.AttributeFilterService'
See what attributes are being released:
./bin/aacli.sh --configDir=conf/ --principal=* --requester=*
You should get something similar to below showing all the attributes you're releasing:
{
"name": "schacHomeOrganization",
"values": [
"uni.ac.uk"
]
},
{
"name": "eduPersonTargetedID",
"values": [
"R45453FGEGE5GEGGSFSDF"
]
}
Restart Tomcat
Check the log for any errors:
systemctl restart tomcat10
Check the IdP process log for any errors and review the warning log for deprecation issues and resolve:
tail -f ./logs/idp-process.log
Check the Warn log:
tail -f ./logs/idp-warn.log
Check the Catalina log (Tomcat):
tail -f /opt/tomcat10/logs/catalina.out
When the IdP has started, you can exit the tail using Control-C.
Linux - Upgrading Shibboleth IdP from v4.3.3 to v5.1.2:
Back up your current system either by taking a snapshot of your virtual machine, or creating tar archives, or a combination of these.
You must ensure that your versions of Java and servlet container (Jetty or Tomcat) meet the system requirements for v5. This is more complex than usual because there are relatively few servlet containers right now that support both Java and Jakarta Servlet specifications, and v4 requires Java while v5 requires Jakarta. At a minimum, it’s a good idea to upgrade a v4 system to Java 17 first, if you have just moved to Java 17 install one of the JavaScript engine plugins (Nashorn or Rhino) be aware that previous versions contained a JavaScript engine, from Java 17 onwards you require a separate JavaScript engine (like Nashorn or Rhino).
You must be on Shibboleth IdP v4.3.3 before proceeding with the upgrade to v5.1.2.
Check the currently installed version
Change to the IdP installed directory and check the currently installed version, do not proceed if it's less that v4.3.3:
cd /opt/shibboleth-idp ./bin/version.sh 4.3.3
Check attributes
Make a note of the attributes that are currently being released, within v4.3.3, for comparison after upgrading:
./bin/aacli.sh --configDir=conf/ --principal=* --requester=*
Note down the output, for comparison later.
Check plugins
Make sure that all plugins are up to date, firstly, see what plugins you have installed:
./bin/plugin.sh -l
For further information about IdP Plugins, please refer to this page: https://shibboleth.atlassian.net/wiki/spaces/IDP5/pages/3199500688/PluginInstallation#Overview
You may have Nashorn v1.1.0 currently installed, this is not compatible with v5.1.2 and needs upgraded to Nashorn v2.0.0 to work.
Upgrade any plugins
Upgrade the Nashorn plugin to v2.0.0 if not already up-to-date:
./bin/plugin.sh -u net.shibboleth.idp.plugin.nashorn INFO - Including auto-located properties in ./../conf/authn/authn.properties INFO - Including auto-located properties in ./../conf/saml-nameid.properties INFO - Including auto-located properties in ./../conf/admin/admin.properties INFO - Including auto-located properties in ./../conf/c14n/subject-c14n.properties INFO - Including auto-located properties in ./../conf/ldap.properties INFO - Including auto-located properties in ./../conf/services.properties INFO - Downloading from HTTPResource [https://shibboleth.net/downloads/identity-provider/plugins/scripting/2.0.0/idp-plugin-nashorn-jdk-dist-2.0.0.tar.gz] INFO - Downloading from HTTPResource [https://shibboleth.net/downloads/identity-provider/plugins/scripting/2.0.0/idp-plugin-nashorn-jdk-dist-2.0.0.tar.gz.asc] INFO - TrustStore does not contain signature 0xxxxxxxxxxx Accept this key: Signature: 0xxxxxxxxxxxxxxxxxx FingerPrint: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx [yN] y INFO - Installing Plugin 'net.shibboleth.idp.plugin.nashorn' version 2.0.0 INFO - Rebuilding /opt/shibboleth-idp/war/idp.war, Version 5.1.2 INFO - Initial populate from /opt/shibboleth-idp/dist/webapp to /opt/shibboleth-idp/webpapp.tmp INFO - Overlay from /opt/shibboleth-idp/dist/plugin-webapp to /opt/shibboleth-idp/webpapp.tmp INFO - Overlay from /opt/shibboleth-idp/edit-webapp to /opt/shibboleth-idp/webpapp.tmp INFO - Creating war file /opt/shibboleth-idp/war/idp.war
Now list the plugins to see if Nashorn v2.0.0 is installed correctly:
./bin/plugin.sh -l INFO - Including auto-located properties in ./bin/../conf/authn/authn.properties INFO - Including auto-located properties in ./bin/../conf/saml-nameid.properties INFO - Including auto-located properties in ./bin/../conf/admin/admin.properties INFO - Including auto-located properties in ./bin/../conf/c14n/subject-c14n.properties INFO - Including auto-located properties in ./bin/../conf/ldap.properties INFO - Including auto-located properties in ./bin/../conf/services.properties Plugin: net.shibboleth.idp.plugin.nashorn Current Version: 2.0.0
Download IdP install files
Change to the src directory and download IdP install files:
Note: Always check to see if there's an update available as the version below may not represent the actual latest version that's available:
https://shibboleth.net/downloads/identity-provider/latest/
cd /usr/local/src curl -O https://shibboleth.net/downloads/identity-provider/latest5/shibboleth-identity-provider-5.1.2.tar.gz curl -O https://shibboleth.net/downloads/identity-provider/latest5/shibboleth-identity-provider-5.1.2.tar.gz.sha256
Check SHA256 message digest matches:
sha256sum -c shibboleth-identity-provider-5.1.2.tar.gz.sha256
Decompress the files
Untar the files and change to the 5.1.2 directory path:
tar xf shibboleth-identity-provider-5.1.2.tar.gz cd shibboleth-identity-provider-5.1.2
Run the installer for v5.1.2
It should detect your current source/install dir and display that on screen as below:
./bin/install.sh Source (Distribution) Directory (press <enter> to accept default): [/usr/local/src/shibboleth-identity-provider-5.1.2] Installation Directory: [/opt/shibboleth-idp] Update from version 4.3.3 to version 5.1.2 Rebuilding /opt/shibboleth-idp/war/idp.war, Version 5.1.2 Initial populate from /opt/shibboleth-idp/dist/webapp to /opt/shibboleth-idp/webpapp.tmp Overlay from /opt/shibboleth-idp/dist/plugin-webapp to /opt/shibboleth-idp/webpapp.tmp Overlay from /opt/shibboleth-idp/edit-webapp to /opt/shibboleth-idp/webpapp.tmp Creating war file /opt/shibboleth-idp/war/idp.war ...done BUILD SUCCESSFUL Total time: 13 seconds
Check the version has been upgraded:
cd /opt/shibboleth-idp ./bin/version.sh 5.1.2
Restart the servlet container
Since this is a major update, it's best to restart everything.
systemctl restart tomcat10
See what attributes are being released:
./bin/aacli.sh --configDir=conf/ --principal=* --requester=*
You should get something similar to below showing all the attributes you're releasing and they should match what the IdP was releasing when at v4.3.3:
{
"name": "schacHomeOrganization",
"values": [
"uni.ac.uk"
]
},
{
"name": "eduPersonTargetedID",
"values": [
"R45453FGEGE5GEGGSFSDF"
]
}
Check the IdP process log for any errors and review the warning log for deprecation issues and resolve:
tail -f ./logs/idp-process.log
Check the Warn log:
tail -f ./logs/idp-warn.log
Check the Catalina log (Tomcat):
tail -f /opt/tomcat10/logs/catalina.out
When the IdP has started, you can exit the tail using Control-C.
Note: The v5.1.2 upgrade will have kept your existing configuration files like attribute-resolver.xml, ldp.properties etc. however you will notice new similarly named files in the conf directory with .idpnew.512 on the end of the file, this is the latest v5.1.2 configuration files, it's highly recommended you migrate your existing configuration settings to the same format as these files, ensuring you have a full backup of your existing configuration before doing so.
IdP Troubleshooting:
Some common errors:
- Has the IdP started correctly and releasing attributes? Connection refused when restarting attribute-resolver Normally caused by a permissions issue with the p12 certificate, check permissions on credentials directory and files.
- Web page showing correct at https://your.idp.url/idp/status with no errors? Servlet exception on web status page You may need to add the Jakarta Servlet 5 file, see IdP browser status error below.
- Any errors showing in the catalina.out (Tomcat) or idp-process.log? (Shibboleth)
- Are directory permissions correct?
IdP browser status error
If you check the IdP web page status https://your.idp.url/idp/status you may notice an error similar to below in the /opt/shibboleth-idp/logs/idp-process.log which points to a servlet exception:
ERROR [jakarta.servlet.ServletException:144] -
jakarta.servlet.ServletException: Handler dispatch failed: java.lang.NoClassDefFoundError:
jakarta/servlet/jsp/jstl/core/Config
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1104)
Caused by: java.lang.NoClassDefFoundError: jakarta/servlet/jsp/jstl/core/Config
at org.springframework.web.servlet.support.JstlUtils.exposeLocalizationContext(JstlUtils.java:103)
Caused by: java.lang.ClassNotFoundException: jakarta.servlet.jsp.jstl.core.Config
at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1353)
It will show in the browser similar to the below:
Web Login Service - Error An error occurred: ServletException
This is because Jakarta Servlet 5 is looking for the Jakarta Standard Tag Library API, to correct this, change to the correct directory and download the following file:
cd /opt/shibboleth-idp/edit-webapp/WEB-INF/lib wget https://repo1.maven.org/maven2/jakarta/servlet/jsp/jstl/jakarta.servlet.jsp.jstl-api/3.0.0/jakarta.servlet.jsp.jstl-api-3.0.0.jar
Rebuild the idp.war file so it contains the new jar file required for the servlet to run correctly:
cd /opt/shibboleth-idp/bin/ ./build.sh
The idp.war file will then be rebuilt in /opt/shibboleth-idp/war containing the newly added jar file. If your current idp.war file is located in /opt/tomcat10/webapps you can copy this new idp.war file over your existing idp.war file.
Windows-specific
Windows platform support
OS Support
Shibboleth IdP v5 requires a 64-bit operating system with Java 17 64-bit installed (32-bit will not work). Supported windows operating systems (from a Java standpoint) are Windows Server 2016 (64-bit, any architecture) or newer. You should be running a current supported Windows environment, search for your environment from here to see life cycle of the product
https://support.microsoft.com/en-gb/lifecycle/search?alpha=Windows
Proposed steps for upgrade from Shibboleth IdP v4:
- Prepare OS:
- Upgrade windows to 2016+
- Fulfil prereqs:
- Upgrade java to 17
- Upgrade to latest Shib v4
- Upgrade Shibboleth
Take snapshots (if VM) and/or ensure you have usable backups before carrying out any changes to your system. There will be several periods of service downtime during the upgrade process (beyond the usual ~20 seconds on unresponsiveness for a service restart).
Windows server
Ensure your OS is Windows server 2016 64-bit or newer. OS upgrades are outside the scope of this document, there are many guides as to how to do this on the internet
Java upgrade - Windows
Upgrading Java
Amazon Corretto 17 for Windows is the fully supported Java Distribution for shibboleth 5 Identity Provider. Installation instructions are available from Amazon Corretto.
In contrast, Shib v4 will run on Java 11 or 17 - the main difference in running Shib v4 on these 2 Java versions is the handling of Javascript within configs (for instance in attribute resolver) – Java 17 will require the Nashorn plugin to be installed - admin command window:
"c:\Program Files (x86)\Shibboleth\IdP)bin\plugin.bat" -I net.shibboleth.idp.plugin.nashorn
On startup, the Shibboleth and Java version will be stated in the idp-process.log file: Shibboleth IdP Version 4.2.1 Java version='17.0.9' vendor='Amazon.com Inc.'
Jetty upgrade for Windows
Version 5 of the IdP requires Jetty v11 or greater. The current situation for Jetty on Windows is that the latest installer uses Jetty v11, although we expect this to updated to Jetty v12 very soon, the recommendation is to use the latest Shibboleth Project provided Jetty installer.
As of January 2024 Jetty v11 has already reached "end of community support" although security updates are still being provided until it is deemed end of life, which we expect will be January 2025, going on previous lifespans.
The Shibboleth project will be releasing an updated Shibboleth IdP Jetty Configuration Baseline (jetty-base) for Jetty 12 on Windows soon. In the meantime, our advice to deployers of Jetty on Windows is that you should be running the latest provided Shibboleth Project Jetty 11 installer, and utilise this until such time as a Jetty 12 installer on Windows becomes available from the Shibboleth Project.
Upgrade to latest Shibboleth v4 (v4.3.3)
Read the entire guide first before proceeding with the upgrade!
Part of the process of upgrading to version 5 of the Shibboleth Identity Provider (IdP) should involve updating to the latest version 4 release.
Shibboleth upgrades direct from v3 to v5 are NOT supported, you must upgrade from v3 to v4 before going on to upgrade to v5. If you're upgrading from Shibboleth v3, please remember that this is a four year old Shibboleth build with many sub version changes, and after upgrading to v4 you MUST address any deprecation warnings before upgrading to v5. Once you have upgraded to v5 you will then need to check for further deprecations.
It's advisable you clone your current production IdP that you plan to upgrade so you can carry out a dummy upgrade in an isolated test environment to address any issues for when you carry out the upgrade on your production IdP.
Check for deprecation warnings
This is especially important if you have recently upgraded from Shibboleth v3. Remember you need to have upgraded to the latest v3 release prior to upgrading to v4, and there is no upgrade path direct from Shibboleth v3 to v5!
After each Shibboleth IdP upgrade you must check for deprecations in the IdP WARN log, this will result in additional support time required from IdP Operators to address warnings before proceeding, otherwise you run the risk of hitting undocumented support issues later. Note that some of these deprecation warnings may only be visible in your servlet container’s log. Restart the Shibboleth IdP service after any changes are made to files.
Of relevance to eduGAIN organisations is warnings as shown below as this is still valid in v5 configs so can be safely ignored.
WARN DEPRECATED xsi:type 'SAML2NameID', (file [C:\Program Files (x86)\Shibboleth\IdP\conf\attribute-resolver.xml]): This will be removed in the next major version of this software; replacement is (none)
Some common v3/v4 deprecated features you may encounter and need addressed below prior to upgrading:
NOTE: Ensure you have a good backup prior to editing any xml files.
Legacy Principal Connector:
WARN [DEPRECATED] - Spring bean 'c14n/LegacyPrincipalConnector', (c14n/subject-c14n.xml): This will be removed in the next major version of this software; replacement is <remove>
or
Cannot resolve reference to bean 'c14n/LegacyPrincipalConnector'
Remove the bean reference for 'c14n/LegacyPrincipalConnector' in C:\Program Files (x86)\Shibboleth\IdP\conf\c14n\subject-c14n.xml.
You can comment this out/remove to clear this deprecation warning:
<ref bean="c14n/LegacyPrincipalConnector" />
New role descriptor filter name:
WARN [DEPRECATED] - xsi:type 'EntityRoleWhiteList', (file [C:\Program Files (x86)\Shibboleth\IdP\conf\metadata-providers.xml]): This will be removed in the next major version of this software; replacement is EntityRole
Edit C:\Program Files (x86)\Shibboleth\IdP\conf\metadata-providers.xml and change 'EntityRoleWhiteList' to 'EntityRole', this is the new role descriptor filter name.
Check the currently installed version
Run an Administrator command prompt window (cmd.exe)
Do not proceed if it's less than v4.3.1:
cd "C:\Program Files (x86)\Shibboleth\IdP\bin\" version.bat"
Download the files
Check if your deployment includes Jetty (does a Jetty directory exist under C:\Program Files (x86)\Shibboleth?)
Download the latest v4 msi
Run the installer for Shibboleth v4.3.3
When you run the installer you will see an option to 'Run As User', it's worth mentioning at this point that if you have not done this previously using the recommended 'Run As User' as a standard user, it may be worth considering this now as by default no user will be specified and the webserver will be run with the default Windows Service account (which is usually highly privileged).'''
Note: When using the 'Run As User' option so that Jetty uses a restricted user account, add the user account to 'Logon as a Service' in the Local Security Policy > Security Setting > Local Policies > User Rights Assignments > 'Logon as a Service' or via Computer/Machine Configuration in a GPO/Azure policy. If you set a restricted user account and don't assign it to the 'Logon as a Service' prior to the install, it won't create the windows service and you'll get an error in the system log of the event viewer similar to:
"The shib_idp service was unable to log on as 'username' with the currently configured password due to the following error: Logon failure: the user has not been granted the requested logon type at this computer"
Run the file below with Administrator privileges, it should detect your current source/install Shibboleth directory:
shibboleth-identity-provider-4.3.3-x64.msi
Note that by default it will install over the top of an existing installation. Make sure you tick the ‘install Jetty’ option if prompted, if you have an existing Jetty install.
After the install completes, restart the Shibboleth 4 IdP service using the standard Windows tools (services.msc).
Check idp-warn.log for any deprecations that need addressed. Restart the Shibboleth IdP service after any changes are made to any files.
Check attributes
Check that attributes are being released correctly after upgrading:
aacli.bat --configDir=conf/ --principal=* --requester=*"
Upgrade to the latest Shibboleth 5
Read the entire guide first before proceeding with the upgrade!
Unlike in Shibboleth v4, the Windows Installation has changed significantly in Shibboleth v5 and is split into two parts – Shibboleth and Jetty. You need to install both of these.
Back up your current system either by taking a snapshot of your physical/virtual machine before attempting any upgrade, and ideally clone your current production IdP, that you plan to upgrade, so you can carry out a test upgrade in an isolated test environment to address any issues for when you carry out the upgrade on your production IdP.
Check idp-warn.log for any deprecations that need addressed. Restart the Shibboleth IdP service after any changes are made to any files.
Download IdP install files
Note: Always check to see if there's an update available as the version below may not represent the latest version available:
Download the msi installer from https://shibboleth.net/downloads/identity-provider/latest5/
The Shibboleth Windows IdP upgrade instructions are available on the Shibboleth Project site for reference: https://shibboleth.atlassian.net/wiki/spaces/IDP5/pages/3199500925/Upgrading
You must be on Shibboleth IdP v4.3.3 before proceeding with the upgrade to v5.1.2 and have addressed any deprecations.
Check the currently installed version
Run an Administrator command prompt window (cmd.exe)
Change to the IdP installed directory and check the currently installed version, do not proceed if it's less that v4.3.3:
Do not proceed if it's less than v4.3.3:
cd "C:\Program Files (x86)\Shibboleth\IdP\bin\" version.bat"
Check attributes
Make a note of the attributes that are currently being released, within v4.3.3, for comparison after upgrading:
aacli.bat --configDir=conf/ --principal=* --requester=*"
Note down the output, for comparison later.
Check plugins
At this point you will need to ensure that all plugins and modules are up to date.
Make sure that all plugins are up to date, firstly, see what plugins you have installed:
cd "C:\Program Files (x86)\Shibboleth\IdP\bin\" plugin.bat -l
For further information about IdP Plugins, please refer to this page: https://shibboleth.atlassian.net/wiki/spaces/IDP5/pages/3199500688/PluginInstallation#Overview
You may have Nashorn v1.1.0 currently installed, this is not compatible with v5.1.2 and needs upgraded to Nashorn v2.0.0 to work.
Upgrade any plugins
Upgrade the Nashorn plugin to v2.0.0 if not already up-to-date:
plugin.bat -u net.shibboleth.idp.plugin.nashorn
Now list the plugins to see if Nashorn v2.0.0 is installed correctly:
plugin.bat -l
Install files
The upgrade to v5 will use your existing installation path, ensure you have a good backup before attempting any upgrade. There will be no options to select in the IdP installer other than Next/Finish. It will also remove any installed jetty-base you may have as part of the IdP install, so make sure you have a backup of your jetty-base in case it fails for any reason.
Note: After you install the IdP msi you will notice there will be no IdP service showing, this is normal and will appear after you install the Jetty msi.
Install v5.1.2 using the following file with Administrator privileges:
shibboleth-idp-5.1.2-x64.msi
It's worth mentioning at this point that if you have not installed your previous Jetty-base using the recommended 'Run As User' as a restricted user, it may be worth considering this now as by default no user will be specified and the webserver will be run with the default Windows Service account (which is usually highly privileged).
Note: When using the 'Run As User' option with a restricted user account, add the user account to 'Logon as a Service' in the Local Security Policy > Security Setting > Local Policies > User Rights Assignments > 'Logon as a Service' or via Computer/Machine Configuration in a GPO/Azure policy. If you set a restricted user account and don't assign it to the 'Logon as a Service' prior to the Jetty install, it won't create the windows service and you'll get an error in the system log of the event viewer similar to:
"The shib_idp service was unable to log on as 'username' with the currently configured password due to the following error: Logon failure: the user has not been granted the requested logon type at this computer"
Install the latest Jetty msi from here: https://shibboleth.net/downloads/identity-provider/jetty-windows/
Do you want to run Jetty using a restricted 'Run As User'? complete the installer wizard.
Now the upgrade should be complete, restart the Shibboleth v5 IdP service using the standard Windows tools (services.msc) and check for deprecation messages in the log files.
Check the IdP process log for any errors and review the warning log for deprecation issues and resolve:
notepad.exe "C:\Program files (x86)\Shibboleth\Idp\logs\idp-process.log"
Check the Warn log:
notepad.exe "C:\Program files (x86)\Shibboleth\Idp\logs\idp-warn.log"
Check the Jetty log:
notepad.exe "C:\Program files (x86)\Shibboleth\Idp\Jetty-base\logs\jetty.log"
Check for deprecation warnings
After each Shibboleth IdP upgrade you must address any deprecations that are highlighted in the IdP WARN log. Note that some of these deprecation warnings may only be visible in your servlet container’s log. Restart the Shibboleth IdP service after any changes are made to any files.
Of relevance to eduGAIN organisations is warnings regarding as shown below this is still valid in v5 configs so can be safely ignored.
WARN DEPRECATED xsi:type 'SAML2NameID', (file [C:\Program Files (x86)\Shibboleth\IdP\conf\attribute-resolver.xml]): This will be removed in the next major version of this software; replacement is (none)
Some deprecations you may encounter and need to be addressed are listed below after upgrading.
NOTE: Ensure you have a good backup before editing any xml files.
Some Deprecations:
Liberty Beans:
WARN [DEPRECATED] - Spring bean 'Liberty.SSOS or Liberty.SSOS.MDDriven', (relying-party.xml): This will be removed in the next major version of this software; replacement is (none)
Edit "C:\Program Files (x86)\Shibboleth\IdP\conf\relying-party.xml" and remove all beans named Liberty.SSOS or Liberty.SSOS.MDDriven to clear this deprecation warning:
X509 Beans:
WARN [DEPRECATED] - Java class 'net.shibboleth.idp.profile.spring.factory.BasicX509CredentialFactoryBean': This will be removed in the next major version of this software; replacement is Parent bean 'shibboleth.BasicX509CredentialFactoryBean'
Edit "C:\Program Files (x86)\Shibboleth\IdP\conf\credentials.xml" and replace all occurrences of:
class="net.shibboleth.idp.profile.spring.factory.BasicX509CredentialFactoryBean"
with:
parent="shibboleth.BasicX509CredentialFactoryBean"
Check attributes
Check that attributes are being released correctly after upgrading:
aacli.bat --configDir=conf/ --principal=* --requester=*"
After installation - Check permissions
By default the Windows Shib IdP install has no restrictive permissions in v5, it shouldn't be put into production without restrictions in place, the SetACL command can be used to constrain permissions. "An account to be given explicit read access to the distribution, and write access to the log folder. This is to allow the web server to be run by a limited permissions user (so the webserver cannot write to the machine), The default is that no user will be specified and the webserver will be run as the default Windows Service account (which is usually highly privileged)."
As highlighted here: https://shibboleth.atlassian.net/wiki/spaces/IDP5/pages/3254484996/SetACLCommand
General
This section contains those steps of the upgrade process that are not platform-dependent: which is to say, those steps that relate to the IdP itself.
SAML Proxying Issue:
If your Shibboleth IdP leverages the SAML proxying feature to pull attributes from Azure AD then your Azure AD metadata may fail to load. If you have this issue then you will need to add the following line to your conf/idp.properties
# opensaml.config.xml.unmarshall.strictMode = false
IdP deployers should use MDQ rather than metadata aggregates
The UK federation publishes metadata via its Metadata Publication Service describing all participating entities. This metadata file (ukfederation-metadata.xml) provides the information required for entities to know how to communicate with each other, and establishes a trust fabric permitting entities to verify each other’s identities. The IdP relies on the regular downloading of this metadata aggregate which is currently (Septermber 2020) over 58MB and will continue to grow with new entities added to the federation. This download is then processed locally within Java, including signature verification, which overall requires a large amount of RAM and CPU resources.
We have a new publication which use a newer protocol called Metadata Query (MDQ) to retrieve metadata. Rather than downloading the whole aggregate XML file, the metadata is looked up on demand by the IdP by querying a UK federation MDQ publication service. Thus, the required CPU and memory footprint is massively reduced.
Further details on MDQ and specifically on configuring metadata providers in the IdP
Note: that deployers who continue to use the federation metadata aggregate will eventually hit either the limits of either CPU and Memory available on the host or limits within the configuration such as Java memory assignment, resulting in a service outage or needing to size-up their environment to cope, and incurring further costs.