IdPv5 upgrade guide

Last Update: 2024-08-16

Introduction

This page contains our guidance for operators of Shibboleth Identity Providers (IdPs) within the UK federation.

Specifically, it is aimed at operators of an existing IdP running the Shibboleth IdP v4 software, that will need to be upgraded to v5.

This upgrade should be completed before September 1, 2024 when the Shibboleth IdP v4 software will reach end of support with the Shibboleth project.

Whilst the UK federation support desk will provide support for operators upgrading their IdP beyond Autumn 2024, we would advise using the following guide, in addition to documentation on the official Shibboleth IdP v5 Wiki, and upgrading as soon as possible, to ensure that you can receive necessary security updates to the software.

For organisations requiring in-depth and hands-on support we offer a Trust and Identity consultancy service

We have made every effort to provide guides that will work for you; however you are encouraged to read in full the guides before starting work, confirm their validity for your deployment and to make backups of your system before beginning.

Using the guide

Linux-specific
Windows-specific
General

�

Linux-specific

Linux platform support

You should be running a current, supported distribution and release of Linux. It can be quite hard to determine when support will end for a given platform. Check https://endoflife.software/operating-systems/ for your platform and release.

You will need to know the Linux distribution and release you are running. Most distributions provide a standard command:

 lsb_release -a

This will provide output similar to:

 Distributor ID:  Ubuntu
 Description:     Ubuntu 22.04.4 LTS
 Release:         22.04
 Codename:	  jammy

If that doesn't provide the information then you may find it by running:

 cat /etc/os-release

Does my Linux distribution supply a recent enough Java and web container?

This guide covers both the Tomcat and Jetty web containers. Different Linux distributions and releases have different versions of Java and Tomcat or Jetty available.

To run IdP5 You will need Java 17 (or later) and either Tomcat 10.1 (or later) or Jetty 11 (or later) so you may need to upgrade various components.

The tables below will allow you to check if your distribution makes suitable versions of the web containers available.

If you're running a generally supported distribution release which doesn't provide the relevant supporting packages then you will need to either upgrade/migrate to one that does or manage the software dependencies yourself.

If you've visited our pages before you may note that CentOS is missing. Cloud providers (eg AWS) along with Rocky are now down-stream distributions of RedHat.

Tomcat

Distibution	Release	Java available	Java suitable	Tomcat available	Suitable	End of life
RedHat	9	17	Yes	10 (manual)	Yes	beyond 2031
Debian	11 bullseye	17	Yes	10 (manual)	Yes	LTS June, 2026
Debian	12 bookworm	17	Yes	10 (package)	Yes	LTS June, 2028
Ubuntu	22.04 LTS	17	Yes	10 (manual)	Yes	April, 2027
Ubuntu	24.04* LTS	17	Yes	10 (package)	Yes	April, 2029

Please note that Ubuntu 24.04 was released in April 2024 and you may wish to have a discussion around your risk appetite in adopting such a new distribution

If you already have a suitable version of Java installed, you can skip ahead to Tomcat upgrade for Linux deployers.

Jetty

It can be tricky obtaining a suitable version of Jetty, the following table describes the situation for major distributions.

Distibution	Release	Java available	Java suitable	Jetty available	Suitable	End of life
RedHat	9	17	Yes	11 (manual)	Yes	beyond 2031
Debian	11 bullseye	17	Yes	11 (manual)	Yes	LTS June, 2026
Debian	12 bookworm	17	Yes	11 (manual)	Yes	LTS June, 2028
Ubuntu	22.04 LTS	17	Yes	11 (manual)	Yes	April, 2027

If you already have a suitable version of Java installed, you can skip ahead to Jetty upgrade for Linux deployers.

Java upgrade - Linux

In general, you should select a Linux distribution and release which provides a supported Java runtime.

Support matrices for popular combinations of platform, Java and web container are given above.

Using the distributions package manager (yum, dnf, apt etc) is the most convenient way to manage the available Java.

The following Java distributions are fully supported:

Amazon Corretto 17 for Linux
Red Hat's OpenJDK 17 for Linux as supplied under Red Hat Enterprise Linux (or compatible rebuild)

Note that this specifically refers to actual Java 17, and not �latest�. Some older versions include �floating� versions but not a specifically supported package for Java 17.

Other Java distributions that are substantially identical or meant to be fully compatible with OpenJDK 17 are of course likely to work, but are officially regarded as unsupported.

First check which version is currently installed:

 $ java -version
openjdk version "11.0.22" 2024-01-16
OpenJDK Runtime Environment (build 11.0.22+7-post-Ubuntu-0ubuntu222.04.1)
OpenJDK 64-Bit Server VM (build 11.0.22+7-post-Ubuntu-0ubuntu222.04.1, mixed mode, sharing)

If you need to upgrade then find the appropriate package name using the package manager's search function:

CentOS / RedHat: yum search openjdk
Debian / Ubuntu: apt-cache search openjdk

When you've found the package, install it using the package manager. Below are some examples but check first:

 # CentOS / RedHat 
 yum install java-17-openjdk

 # Debian / Ubuntu
 apt-get install openjdk-17-jdk

Verify the new version is being used by default:

 $ java -version
 openjdk version "17.0.10" 2024-01-16
 OpenJDK Runtime Environment (build 17.0.10+7-Ubuntu-122.04.1)
 OpenJDK 64-Bit Server VM (build 17.0.10+7-Ubuntu-122.04.1, mixed mode, sharing)

If the install process has not enabled by default then you will need to use the update-alternatives system to select it:

 update-alternatives --config java

Remember to check your JAVA_HOME is set correctly after upgrading.

Note: Java 17 does not include a JavaScript Engine like it did in earlier versions, you will need a plugin like Nashorn, there are instructions how to do this in the Shibboleth IdP v4.3.3 -> v5.1.x upgrade guide on this page.

�

Tomcat upgrade for Linux deployers

Tomcat upgrade/install

Shibboleth v5 requires Tomcat v10 and above.

There are incompatibilities between Shibboleth v4.3.1 and Tomcat v10, so if you start this process you are strongly urged to snapshot your working VM before attempting the upgrade (or better, if you have a development or staging environment, perform the update there first)

If your distribution has the required version of Tomcat available then your first choice should be that package. Please see your distribution's documentation on how to install or upgrade packages.

However, should you have Tomcat installed from your distribution's package manager and this version is not appropriate for Shibboleth IdP v5, you will need to manually install the latest version (you can of course run the two versions side by side but you will need to put the configuration and installation and logs in their own directories in order to avoid clashes)

eg Tomcat v9 directories named "tomcat9" and Tomcat v10 directories named "tomcat10".

Manual installation of Tomcat

Should your distribution not provide an appropriate Tomcat package then you will need to install and maintain this yourself. The procedure below is just one way of doing this and may need altering to suit your environment.

Note

Before doing this, ensure you have a supported version of Java available.

Overview

We're assuming you are upgrading from v4 and Tomcat9

We will be using Tomcat10 as both webserver and servlet container, removing the need for Apache to act as a proxy.

Download the latest version from Tomcat 10 and verify
Decompress the archive
Create a tomcat user if required
Prepare the new directory tree (move bundled directories and symlink to system locations)
Create master symlink
Create service
Configure Tomcat
Disable (or uninstall) Tomcat9 (if it's already installed)
Start Tomcat service

Details

Download the latest version

Check the latest version of Tomcat 10 available. Download the package and verify the signature to ensure it's not been tampered with:

For example:

 $ cd /usr/local/src
 $ curl -O https://dlcdn.apache.org/tomcat/tomcat-10/v10.1.19/bin/apache-tomcat-10.1.19.tar.gz
 $ curl -O https://downloads.apache.org/tomcat/tomcat-10/v10.1.19/bin/apache-tomcat-10.1.19.tar.gz.sha512

 $ sha512sum -c apache-tomcat-10.1.19.tar.gz.sha512

 apache-tomcat-10.1.19.tar.gz: OK

Decompress the archive

 cd /opt
 tar xfz /usr/local/src/apache-tomcat-10.1.19.tar.gz

If required, create a tomcat user

Check if a tomcat user is already available:

 getent passwd tomcat

If nothing is returned then you will need to create a user to run tomcat as. This can vary between distributions but generally looks like:

 useradd --system --no-create-home --home-dir /opt/tomcat10 tomcat

Prepare the directory tree

To make upgrades easier, move the components that need to persist across Tomcat upgrades out to their traditional system locations and replace them with symlinks. Lastly, ensure the required directories are writeable by the tomcat user.

 mkdir -p /var/cache/tomcat10 /var/lib/tomcat10 /var/log/tomcat10 /etc/tomcat10

 cd /opt/apache-tomcat-10.{rest of directory name}

 mv -i conf/* /etc/tomcat10/
 mv -i logs /var/log/tomcat10/
 mv -i temp /var/cache/tomcat10/
 mv -i webapps /var/lib/tomcat10/
 mv -i work /var/cache/tomcat10/

 ln -s /etc/tomcat10 conf
 ln -s /var/log/tomcat10 logs
 ln -s /var/cache/tomcat10/temp temp
 ln -s /var/lib/tomcat10/webapps webapps
 ln -s /var/cache/tomcat10/work work

 chown -R tomcat /var/log/tomcat10 /var/cache/tomcat10 /var/lib/tomcat10 /etc/tomcat10

 chown -R tomcat /opt/apache-tomcat-10.{rest of directory name}

note that if you have an existing tomcat user on your system that is not "tomcat" - replace with that user in the above chown command.

Create the master symlink

To make upgrades easier, create a symlink to the current version:

 cd /opt
 ln -s apache-tomcat-10.{rest of directory name} tomcat10

Create the Tomcat service

Most modern Linux distributions use systemd to manage services. Some distributions use "RC" scripts instead.

To see if your system uses systemd, run:

 systemctl status

If this returns information about running services then you can add a new service by creating a new service unit file.

 nano /etc/systemd/system/tomcat10.service

Make the contents of the file:

 [Unit]
 Description=Tomcat10
 After=syslog.target network.target

 [Service]
 Type=forking

 Environment="JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64"
 Environment="JAVA_OPTS=-Djava.security.egd=file:///dev/urandom"
 Environment="CATALINA_BASE=/opt/tomcat10"
 Environment="CATALINA_HOME=/opt/tomcat10"
 Environment="CATALINA_PID=/opt/tomcat10/temp/tomcat.pid"
 Environment="CATALINA_OPTS=-Xms512M -Xmx1024M -server -XX:+UseParallelGC"

 ExecStart=/opt/tomcat10/bin/startup.sh
 ExecStop=/opt/tomcat10/bin/shutdown.sh

 RestartSec=10
 Restart=always

 User=tomcat

 AmbientCapabilities=CAP_NET_BIND_SERVICE


 ReadWritePaths=/opt/shibboleth-idp


 [Install]
 WantedBy=multi-user.target

NOTE that JAVA_HOME in the above can be found by running: ls -l /etc/alternatives/java and taking the path=].
NOTE that again, if a preexisting tomcat user is different to "tomcat", replace in the above text.
NOTE the line above AmbientCapabilities=CAP_NET_BIND_SERVICE which tells systemd to allow Tomcat10 to access privileged network ports.

Save and quit (Ctrl-X and follow the prompts).

Next, stop Apache HTTP Server:

 apachectl stop

Next, tell systemd to enable your Tomcat v10 installation. You may also want to disable any existing Tomcat v9 installation:

 systemctl enable tomcat10
 systemctl disable tomcat9

You may need to uninstall Tomcat9

On Ubuntu (and possibly other distributions) this is important as it removes Tomcat/Apache library(-ies) which Tomcat10 may complain are out of date. Do this if disabling doesn't work in the first instance.

Lastly tell systemd to reload its configuration:

 systemctl daemon-reload

Configure Tomcat

The Tomcat configuration requires a few tweaks to support the IdP.

Note that we're assuming an installation of Tomcat v10 (note the directory names in the following). You will need to alter the directory names if you've chosen a different naming convention for any reason.

Note The following moves away from using Apache as a proxy for web requests on port 443 being passed into Tomcat and onto the IdP. The following uses Tomcat as the Web server and Servlet container. This means that Apache will not be able to run alongside.

idp.xml

 mkdir -p /etc/tomcat10/Catalina/localhost
 nano /etc/tomcat10/Catalina/localhost/idp.xml

Make the contents of this file:

 <Context docBase="/opt/shibboleth-idp/war/idp.war"
          privileged="true"
          antiResourceLocking="false"
          swallowOutput="true">

    <!-- Work around lack of Max-Age support in IE/Edge for Tomcat 8.0.x -->
    <CookieProcessor alwaysAddExpires="true" />
    <Parameter name="idp.home" value="/opt/shibboleth-idp" override="true" />
 </Context>

Save and quit.

catalina.properties

Add the following to the end of this file:

 # HTTP connector
 # Allows use of default IdP command line tools.
 # tomcat.http.host=127.0.0.1
 # tomcat.http.port=80

 # The HTTPS connector
 # The interface to listen on. To listen on all interfaces, set tomcat.host = 0.0.0.0
 tomcat.https.host=0.0.0.0
 tomcat.https.port=443

 # pem file variables
 tomcat.https.pem.certificateFile=<ssl-cert-directory>/<idp certificate file.crt>
 tomcat.https.pem.certificateKeyFile=<ssl-cert-directory>/<idp certificate's key file.key>
 tomcat.https.pem.certificateChainFile=<ssl-cert-directory>/<chain file.pem>

TLS certificate

You will note above that we specify the web-facing TLS certificate in this part of the Tomcat configuration (since we're not using Apache). Modern versions of Tomcat (including 10) allow use of PEM files just as one uses with Apache. When expanding the variables above you must make sure that the tomcat user can read the three files.

server.xml

You need to add at least a connector for https (port 443). If you wish you can also add a connector for http (port 80). We suggest this as optional in case you're using automation to fetch TLS certificates and that process causes a temporary webserver to be spun up at renewal time.

Paste the following within the <Service name="Catalina"> block, at its start:

 <!-- Allows access to port https - port 443. -->
 <Connector address="${tomcat.https.host}"
           port="${tomcat.https.port}"
           protocol="org.apache.coyote.http11.Http11NioProtocol"
           SSLEnabled="true"
           scheme="https"
           secure="true">
     <SSLHostConfig ciphers="ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-CHACHA20-POLY1305"
                   disableSessionTickets="true"
                   honorCipherOrder="false"
                   protocols="TLSv1.2, TLSv1.3">
            <Certificate certificateFile="${tomcat.https.pem.certificateFile}"
                         certificateKeyFile="${tomcat.https.pem.certificateKeyFile}"
                         certificateChainFile="${tomcat.https.pem.certificateChainFile}" />
     </SSLHostConfig>
 </Connector>

If you do wish to listen on port 80, you need to decide whether you want the machine to to listen on localhost only - this would allow use of the Shibboleth command line tools to check functionality of the IdP. See the section above for catalina.properties and note the host is set to 127.0.0.1 in our example.

The connector block for http (port 80) is as follows:

 <!-- Allows access to port http - port 80. -->
 <Connector address="${tomcat.http.host}"
                 port="${tomcat.http.port}"
                 protocol="HTTP/1.1"
                 connectionTimeout="20000"
                 redirectPort="${tomcat.https.port}"/>

You should also comment out the other Connector blocks.

Setting the default page for contexts outside /idp/ along with http error trapping

 cd /opt/tomcat10/webapps
cp -r ROOT ROOT-vanilla
cd ROOT
rm * # note this will deliberately not remove the WEB-INF directory that's in /opt/tomcat10/webapps/ROOT

Create an index.html that looks like the page at /idp/

Next, edit the web.xml file in WEB-INF:

 vim WEB-INF/web.xml

Remove the following text:

 <display-name>Welcome to Tomcat</display-name>
  <description>
     Welcome to Tomcat
  </description>

Add the following before the final closing tag in the file:

 <!-- Other HTTP errors handled here -->
    <error-page>
            <error-code>404</error-code>
            <location>/index.html</location>
    </error-page>
    <error-page>
            <error-code>403</error-code>
            <location>/index.html</location>
    </error-page>
    <error-page>
            <error-code>500</error-code>
            <location>/index.html</location>
    </error-page>

Start the Tomcat service

 systemctl start tomcat10

Check the startup logs:

journalctl -f
cat /var/log/tomcat10/catalina.out

Tomcat Troubleshooting

If Tomcat cannot deploy the warfile with the set docBase in the idp.xml file in /etc/tomcat10/Catalina/localhost/idp.xml you can try copying the idp.war file into the /opt/tomcat10/webapps directory to see if it will load, this will let you know if there is a permissions issue accessing the warfile directly directly from the docBase path.

Check to see if the idp.war loads and is referenced in /opt/tomcat10/logs/catalina.out like below.

 INFO [main] org.apache.catalina.startup.HostConfig.deployWAR Deploying web application archive 
 [/var/lib/tomcat10/webapps/idp.war]

Note: /var/lib/tomcat10/webapps was symlinked earlier from /opt/tomcat10/webapps

�

Upgrade of manually installed Tomcat - beyond initial v9 -> v10

Once you're managing Tomcat as described above, the upgrade process is:

Download the latest version (check Shibboleth IdP requirements) from Tomcat and verify
Decompress the archive
Prepare the new directory tree (remove bundled directories and symlink system locations)
Replace master symlink
Restart Tomcat

This may look like:

 # Download the latest version
 cd /usr/local/src
 curl -O {tar.gz file}
 curl -O {tar.gz.sha512 file}

 sha512sum -c {tar.gz.sha512 file}

 # Decompress the archive
 cd /opt
 tar xfz /usr/local/src/{tar.gz file}

 # Prepare the new directory tree
 cd {directory that you uncompress the tar.gz file into}
 mkdir ~/tomcat10-backup-1970-01-01
 mv -fr conf logs temp webapps work ~/tomcat10-backup--1970-01-01
 ln -s /etc/tomcat10 conf
 ln -s /var/log/tomcat10 logs
 ln -s /var/cache/tomcat10/temp temp
 ln -s /var/lib/tomcat10/webapps webapps
 ln -s /var/cache/tomcat10/work work
 chown -R tomcat .

 # Replace master symlink
 cd /opt
 ls -l tomcat10 # note current version
 lrwxrwxrwx. 1 root root YY MMM DD HH:MM tomcat10 -> apache-tomcat-10.something
 rm -f tomcat10
 ln -s {directory that you uncompressed the tar.gz file into} tomcat10

 # Restart Tomcat
 systemctl restart tomcat10

�

Jetty upgrade for Linux deployers

Shibboleth IdP v5 requires Jetty v11 or greater. The Shibboleth project has released an IdP Jetty Configuration Baseline (jetty-base) supporting Jetty v12, and our advice to deployers of Jetty with IdP v5 on Linux is to upgrade to Jetty v12 to ensure current security updates and IdP interoperability.

The recommended container implementation by the Shibboleth project is Jetty, and all development and most testing time by the core project team is confined to the Jetty platform. At present, Jetty v11 or v12 can be used, however, as of January 2024 Jetty v11 has already reached 'end of community support' although security updates will still be provided for now until it's deemed end of life, which we expect will be in January 2025, going on previous lifespans.

Current and up-to-date Jetty packages, such as Jetty v12 tend not to be available for the majority of distributions directly from their main repositories for numerous reasons, because of this a manual installation of Jetty 12 will be required.

You can contact the UK federation service desk at any time for advice with any Jetty upgrade queries.

Note: Back-channel support is no longer supported in Jetty 12!

Manual installation of Jetty 12

Note: Before attempting the Jetty upgrade, ensure you have a supported version of Java available.

Overview

Back up your current system!
Download the latest version of Jetty 12 and verify
Decompress the archive for Jetty
Download the IdP jetty-base files
Decompress the archive for jetty-idp-base
Configure Jetty Modules and JVM Settings (start.ini)
Configure HTTP/HTTPS Connectors/Certificates
Configure IdP Context Descriptors (idp.xml)
Configure Jetty Logging
Add the Jetty Service (if this is a fresh IdP install)

Back up your current system

Back up your system either by taking a snapshot of your virtual machine, or by creating tar archives, or a combination of these.

Download the latest version of Jetty 12

Ensure you download the latest Jetty 12 version to ensure security and interoperability with IdP v5: https://jetty.org/download.html

Change to the /usr/src dir and download Jetty 12:

Note: The URLs below may not reflect the current version of Jetty, always ensure you are downloading the latest version!

 cd /usr/src
 curl -O https://repo1.maven.org/maven2/org/eclipse/jetty/jetty-home/12.0.10/jetty-home-12.0.10.tar.gz
 curl -O https://repo1.maven.org/maven2/org/eclipse/jetty/jetty-home/12.0.10/jetty-home-12.0.10.tar.gz.sha1

Check the message digest match:

 sha1sum jetty-home-12.0.10.tar.gz
 cat jetty-home-12.0.10.tar.gz.sha1

Decompress the archive for Jetty 12

Untar the Jetty files:

 tar xf jetty-home-12.0.10.tar.gz

Move the dir to where you want it e.g. /opt:

 mv jetty-home-12.0.10 /opt/jetty-home

Download the IdP Jetty Base using git

Get the latest IdP jetty-base files for the Shibboleth IdP using git, checkout branch 12 and copy to /opt/jetty-base, then switch to that directory:

 cd /usr/src
 git clone https://git.shibboleth.net/git/java-idp-jetty-base
 cd java-idp-jetty-base
 git checkout 12
 cp -r src/main/resources/jetty-base /opt/jetty-base
 cd /opt/jetty-base

Set your JETTY_HOME and JETTY_BASE:

 export JETTY_HOME=/opt/jetty-home
 export JETTY_BASE=/opt/jetty-base

We strongly recommend placing all IdP-specific Jetty configuration under JETTY_BASE to facilitate Jetty upgrades. Do not place that directory inside the IdP installation directory; they can be siblings as desired. Note that Jetty 12 requires the recommended home/base split; the distribution itself is always considered read-only.

Configure Jetty Modules and JVM Settings

The bulk of the configuration is established by setting properties in "ini" files that are combined in the start.d directory. Some of the properties are defined by Jetty and configure built-in modules and others are specific to the IdP and configure the custom module we created.

Add the following to a new file in the start.d directory or check the following matches the current start.ini file: /opt/jetty-base/start.d/start.ini:

 # Any other required Jetty modules...

# Allows setting Java system properties (-Dname=value)
# and JVM flags (-X, -XX) in this file
# NOTE: spawns child Java process
--exec
# Uncomment if IdP is installed somewhere other than /opt/shibboleth-idp
#-Didp.home=/path/to/shibboleth-idp
# Maximum amount of memory that Jetty may use, at least 1.5G is recommended
# for handling larger (> 25M) metadata files but you will need to test on
# your particular metadata configuration. If MDQ metadata on demand is used,
# requirements may be much lower.
-Xmx1500m
# Prevent blocking for entropy.
-Djava.security.egd=file:/dev/urandom
# Set Java tmp location
-Djava.io.tmpdir=tmp

The start.d/idp.ini file that configures the �idp� module contains not only the bulk of the basic settings needed but is also a place you can add your own settings to control JVM startup, change which logging module is used, etc. Or you can continue to use a separate start.ini file as above.

Configure HTTP/HTTPS Connectors/Certificates

Open /opt/jetty-base/start.d/idp.ini

Copy over your cert from the old jetty credentials location directory to /opt/jetty-base/credentials/ and add passwords and correct location to your p12 cert in the idp.ini file. The remaining default settings should be correct for most installs.

The basic HTTP/HTTPS port, address, etc. configuration can be handled within the custom "idp" module and the /opt/jetty-base/start.d/idp.ini property file. The idp.ini example below shows some of the basic properties you can use to configure networking and TLS credentials.

Note: Back-channel support is no longer supported in Jetty 12!

An example idp.ini file for reference:

  # --------------------------------------- 
# Module: idp
# Shibboleth IdP
# --------------------------------------- 
--module=idp
## Keystore file path (relative to $jetty.base)
jetty.sslContext.keyStorePath=../credentials/idp-userfacing.p12
## Truststore file path (relative to $jetty.base)
jetty.sslContext.trustStorePath=../credentials/idp-userfacing.p12
## Keystore type
jetty.sslContext.keyStoreType=PKCS12
## Truststore type and provider
jetty.sslContext.trustStoreType=PKCS12
## Keystore password
jetty.sslContext.keyStorePassword=changeit
## Truststore password
jetty.sslContext.trustStorePassword=changeit
## KeyManager password
jetty.sslContext.keyManagerPassword=changeit
## Deny SSL renegotiation
jetty.sslContext.renegotiationAllowed=false
## Connector host/address to bind to
# jetty.ssl.host=0.0.0.0
## Connector port to listen on
jetty.ssl.port=443
# Allows use of default IdP command line tools.
jetty.http.host=127.0.0.1
jetty.http.port=8080

The TLS credential example relies on a PKCS12 ? file containing the X.509 certificate and private key used to secure the HTTPS channel that users access during authentication and other browser-based message exchanges involving the IdP. This is generally the one you get from a browser-compatible CA, and the example shows it being loaded from a directory inside the JETTY_BASE tree. 8080 is the http port for Jetty (jetty.http.port=8080), It's also okay to leave this unset. That change will confine Jetty to listen locally only rather than over the network.

Configure IdP Context Descriptor

In order to deploy the IdP war file, Jetty must be informed of the location of the IdP warfile. You can of course change the location of this warfile to match your IdP install path.

You need to be careful here as the context descriptor class specified will be specific to the version of Jetty you're running, so do not copy over old jetty-base idp.xml files (this is not recommended to do at all as these files will change over time), you may want to adjust the path to point to your idp.war file, the default is normally /opt/shibboleth-idp/war/idp.war, check the context path where the war file will be launched from matches what's in this idp.xml file. Check that permissions are set correctly for the path so Jetty can get to it. If the idp.war file does not appear to be running for you, try copying the war file to /opt/jetty-base/webapps, if it runs from there then there are permission issues to the default path specified in the idp.xml file.

Open /opt/jetty-base/webapps/idp.xml and check the path to your warfile is correct.

 <?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">
%comment%  ===============================================================  %%
%comment%  Configure the Shibboleth IdP webapp                              %%
%comment%  ===============================================================  %%
<Configure class="org.eclipse.jetty.ee9.webapp.WebAppContext">
  <Set name="war"><SystemProperty name="idp.home" default="/opt/shibboleth-idp" />/war/idp.war</Set>
  <Set name="contextPath"><SystemProperty name="idp.context.path" default="/idp" /></Set>
  <Set name="extractWAR">false</Set>
  <Set name="copyWebDir">false</Set>
  <Set name="copyWebInf">true</Set>
</Configure>

Jetty Logging

The recommended approach is to use logback for all Jetty logging, the logging module should have already been added to idp.ini, however, you will need to run a command to download the appropriate files from the Maven repo to ensure you have the correct libraries for logging to work.

Download dependency files for logging:

 cd /opt/jetty-base

 java -jar /opt/jetty-home/start.jar --add-module=logging-logback

 ALERT: There are enabled module(s) with licenses.
The following 1 module(s):
 + contains software not provided by the Eclipse Foundation!
 + contains software not covered by the Eclipse Public License!
 + has not been audited for compliance with its license

 Module: logging-logback
  + Logback: the reliable, generic, fast and flexible logging framework.
  + Copyright (C) 1999-2012, QOS.ch. All rights reserved.
  + This program and the accompanying materials are dual-licensed under
  + either:
  + the terms of the Eclipse Public License v1.0
  + as published by the Eclipse Foundation:
  + http://www.eclipse.org/legal/epl-v10.html
  + or (per the licensee's choosing) under
  + the terms of the GNU Lesser General Public License version 2.1
  + as published by the Free Software Foundation:
  + http://www.gnu.org/licenses/old-licenses/lgpl-2.1.html

Proceed (y/N)? Y
INFO  : mkdir ${jetty.base}/start.d
INFO  : logging-logback initialized in ${jetty.base}/start.d/logging-logback.ini
INFO  : resources       transitively enabled
INFO  : logging/slf4j   dynamic dependency of logging-logback
INFO  : mkdir ${jetty.base}/resources
INFO  : copy ${jetty.home}/modules/logging/logback/resources/logback.xml to ${jetty.base}/resources/logback.xml
INFO  : mkdir ${jetty.base}/lib/logging
INFO  : download https://repo1.maven.org/maven2/ch/qos/logback/logback-classic/1.5.6/logback-classic-1.5.6.jar to ${jetty.base}/lib/logging/logback-classic-1.5.6.jar
INFO  : download https://repo1.maven.org/maven2/ch/qos/logback/logback-core/1.5.6/logback-core-1.5.6.jar to ${jetty.base}/lib/logging/logback-core-1.5.6.jar
INFO  : Base directory was modified

Jetty will populate the /opt/jetty-base/lib/logging dir with the two logback libraries required. It's strongly advisable to check the signatures of the files you downloaded, as below!

 curl -O https://repo1.maven.org/maven2/ch/qos/logback/logback-core/1.5.6/logback-core-1.5.6.jar.sha1
sha1sum /opt/jetty-base/lib/logging/logback-core-1.5.6.jar
cat /opt/jetty-base/lib/logging/logback-core-1.5.6.jar.sha1

sha1sum /opt/jetty-base/lib/logging/logback-classic-1.5.6.jar
curl -O https://repo1.maven.org/maven2/ch/qos/logback/logback-classic/1.5.6/logback-classic-1.5.6.jar.sha1
sha1sum /opt/jetty-base/lib/logging/logback-classic-1.5.6.jar
cat /opt/jetty-base/lib/logging/logback-classic-1.5.6.jar.sha1

Open /opt/jetty-base/start.d/idp.ini

Add this to the head of your idp.ini, if it's not already there:

 --module=logging-logback

A sample logback configuration for both debug and request logging is provided in /opt/jetty-base/resources/logback.xml if you need to amend anything related to Jetty logging.

Disable Jetty directory browsing

Jetty has had vulnerabilities in the past related to directory browsing and it's recommended to disable directory browsing on Jetty to mitigate any future vulnerabilities around this, as recommended here. There are a few different ways this can be achieved, but one method that's fairly self-contained within the IdP footprint is to modify web.xml.

You can place the below markup at the top of the file under '<display-name>' markup in the web.xml file. e.g. copy the original version from /opt/shibboleth-idp/dist/webapp/WEB-INF/web.xml to /opt/shibboleth-idp/edit-webapp/WEB-INF/web.xml) after modifying the web.xml file, rebuild the war file so this is encompassed into the generated idp.war output file.

Note: the servlet class may differ depending what Jetty version you're running, but should work fine using Jakarta EE 9 servlet in Jetty v12 as below.

  <servlet>
    <servlet-name>default</servlet-name>
    <servlet-class>org.eclipse.jetty.ee9.servlet.DefaultServlet</servlet-class>
    <init-param>
      <param-name>dirAllowed</param-name>
      <param-value>false</param-value>
    </init-param>
    <load-on-startup>0</load-on-startup>
  </servlet>

Rebuild the war file:

 cd /opt/shibboleth-idp/bin/
 ./build.sh

Add the Jetty Service

The following is a basic example of a systemd service file for starting and stopping the jetty service. You'll need to ensure it's enabled at start-up and that any previous versions of Jetty don't start. If you don't already have a jetty.service file at /etc/systemd/system/jetty.service you can create one as below, note that the Start command below includes the logback configuration File path.

Use POSIX Ambient capabilities to allow use of privileged ports by an unprivileged process, e.g. when using systemd by setting AmbientCapabilities ?=CAP_NET_BIND_SERVICE as can be seen in the jetty.service file below.

Note, you will also need to a non-root user account to include in the service file.

  [Unit]
 Description=Jetty provides a Web server and javax.servlet container, plus support for HTTP/2, WebSocket, OSGi, JMX, JNDI, JAAS and many other integrations.
 After=network.target

 [Service]
 WorkingDirectory=/opt/jetty-base
 ExecStart=/usr/bin/java -Djdk.tls.ephemeralDHKeySize=2048 -Dlogback.configurationFile=resources/logback.xml -jar /opt/jetty-home/start.jar jetty.base=/opt/jetty-base lib=/opt/jetty-home/lib

 User=jetty

 AmbientCapabilities=CAP_NET_BIND_SERVICE

 [Install]
 WantedBy=multi-user.target

Set the minimum amount of directory/file permissions required for Jetty and always ensure you are not running Jetty as root!

To start the service:

 systemctl start jetty

To enable the service at boot:

 systemctl enable jetty

Jetty Troubleshooting

Nothing appears to load and keep getting 503 errors restarting attribute-resolver service

Check to see if the warfile path is set correctly in the idp.xml context descriptor file, the path will be broken up into the default path and warfile directory path like below:

 <Set name="war"><SystemProperty name="idp.home" default="/opt/shibboleth-idp" />/war/idp.war</Set>

- Check your permissions are set correctly.

- Check the log at: /opt/jetty-base/logs/jetty.log to see if there are any clues as to why Jetty is not starting if not related to the warfile path.

Parent for temp dir not configured correctly

The Java -Djava.io.tmpdir=tmp temp directory in start.ini setting may or may not work depending on your Java setup (ensure your JAVA_HOME is set correctly), you may see an error in jetty.log similar to below, you can get around this by setting an absolute path to the temp directory:

 java.lang.IllegalStateException: Parent for temp dir not configured correctly: writeable=false

�

Linux - Upgrading Shibboleth

Linux - Upgrading Shibboleth IdP from v4.3.1 to v4.3.3

Before you can upgrade Shibboleth IdP to v5.1.x you must upgrade to v4.3.3, this update was released in March 2024 and must be installed prior to proceeding to the main v5.1.x update.

Shibboleth IdP V4 software will leave support on September 1, 2024 and it's important that after you upgrade to v4.3.3 you upgrade to v5.1.x before this time.'''

After the upgrade to v4.3.3 you will be required to resolve deprecation warnings. Note that some of these warnings may only be visible in your servlet container�s log. This is due to the order of operations; the IdP�s logging service cannot be started until the global Spring context is running, and some of the configuration triggering warnings may be in that root context.

There is one set of warnings that are an exception to this: the undocumented �SAML2NameID� and �SAML1NameID� attribute encoders produce warnings indicating they were removed from the next major version.

 DEPRECATED xsi:type ''''SAML2NameID'''', (file [/opt/shibboleth-idp/conf/attribute-resolver.xml]): 
 This will be removed in the next major version of this software; replacement is (none)

In fact they were not, and their use is still valid and there are no explicit plans to remove them.

This upgrade assumes Ubuntu 22.04 LTS and Tomcat10 are installed.

It may also be useful to enable DEBUG for the log files, to assist with any troubleshooting you need to do, or if you need to send us log files to assist you with the upgrade.

 /opt/shiboleth-idp/conf/idp.properties

Change from INFO to DEBUG:

 idp.loglevel.idp=DEBUG
 idp.loglevel.ldap=DEBUG
 idp.loglevel.messages=DEBUG
 idp.loglevel.opensaml=DEBUG

Back up your current system either by taking a snapshot of your virtual machine, or creating tar archives, or a combination of these.

Check the currently installed version

 cd /opt/shibboleth-idp
 ./bin/version.sh
 4.3.1

Download the files

 cd /usr/local/src
 curl -O https://shibboleth.net/downloads/identity-provider/latest4/shibboleth-identity-provider-4.3.3.tar.gz
 curl -O https://shibboleth.net/downloads/identity-provider/latest4/shibboleth-identity-provider-4.3.3.tar.gz.sha256

Check SHA256 message digest matches:

 sha256sum -c shibboleth-identity-provider-4.3.3.tar.gz.sha256

Decompress the files

Untar the files and change to the directory:

 tar xf shibboleth-identity-provider-4.3.3.tar.gz
 cd shibboleth-identity-provider-4.3.3

Run the installer for v4.3.3

It should detect your current source/install dir and display that on screen as below:

 ./bin/install.sh

 Source (Distribution) Directory (press <enter> to accept default): [/usr/local/src/shibboleth-identity-provider-4.3.3]

 Installation Directory: [/opt/shibboleth-idp]

 Update from version 4.3.1 to version 4.3.3
 Rebuilding /opt/shibboleth-idp/war/idp.war, Version 4.3.3
 Initial populate from /opt/shibboleth-idp/dist/webapp to /opt/shibboleth-idp/webpapp.tmp
 Overlay from /opt/shibboleth-idp/dist/plugin-webapp to /opt/shibboleth-idp/webpapp.tmp
 Overlay from /opt/shibboleth-idp/edit-webapp to /opt/shibboleth-idp/webpapp.tmp
 Creating war file /opt/shibboleth-idp/war/idp.war
 ...done

Check the version has been upgraded

 cd /opt/shibboleth-idp
 ./bin/version.sh
 4.3.3

Although this is a minor update, please check that everything still works before proceeding to the v5.1.x upgrade.

Reload Attribute Resolver service

This is so there are no issues being flagged on-screen or in the log files

 ./bin/reload-service.sh -id shibboleth.AttributeResolverService

If reloaded correctly you should see:

 Configuration reloaded for 'shibboleth.AttributeResolverService'

Reload Attribute Filter service

Check what attributes are being released and that there are no issues being flagged on-screen or in the log files:

 ./bin/reload-service.sh -id shibboleth.AttributeFilterService

If reloaded correctly you should see:

 Configuration reloaded for 'shibboleth.AttributeFilterService'

See what attributes are being released:

 ./bin/aacli.sh --configDir=conf/ --principal=* --requester=*

You should get something similar to below showing all the attributes you're releasing:

 {
    "name": "schacHomeOrganization",
    "values": [
        "uni.ac.uk"
    ]
  },


  {
    "name": "eduPersonTargetedID",
    "values": [
        "R45453FGEGE5GEGGSFSDF"
    ]
  }

You can also test your IdP against the UK federation Test SP to ensure it's releasing attributes correctly.

Restart Tomcat

Check the log for any errors:

 systemctl restart tomcat10

Check the IdP process log for any errors and review the warning log for deprecation warnings and resolve:

 tail -f ./logs/idp-process.log

Check the Warn log:

 tail -f ./logs/idp-warn.log

Check the Catalina log (Tomcat):

 tail -f /opt/tomcat10/logs/catalina.out

When the IdP has started, you can exit the tail using Control-C.

Linux - Upgrading Shibboleth IdP from v4.3.3 to v5.1.x:

Back up your current system either by taking a snapshot of your virtual machine, or creating tar archives, or a combination of these.

You must ensure that your versions of Java and servlet container (Jetty or Tomcat) meet the system requirements for v5. This is more complex than usual because there are relatively few servlet containers right now that support both Java and Jakarta Servlet specifications, and v4 requires Java while v5 requires Jakarta. At a minimum, it�s a good idea to upgrade a v4 system to Java 17 first, if you have just moved to Java 17 install one of the JavaScript engine plugins (Nashorn or Rhino) be aware that previous versions contained a JavaScript engine, from Java 17 onwards you require a separate JavaScript engine (like Nashorn or Rhino).

You must be on Shibboleth IdP v4.3.3 before proceeding with the upgrade to v5.1.x.

Check the currently installed version

Change to the IdP installed directory and check the currently installed version, do not proceed if it's less that v4.3.3:

 cd /opt/shibboleth-idp
 ./bin/version.sh
 4.3.3

Check attributes

Make a note of the attributes that are currently being released, within v4.3.3, for comparison after upgrading:

 ./bin/aacli.sh --configDir=conf/ --principal=* --requester=*

Note down the output, for comparison later.

Download IdP install files

Change to the src directory and download IdP install files:

Note: Always check to see if there's an update available as the version below may not represent the actual latest version that's available and the subversion will show as 5.1.x replace this with the latest verison:

https://shibboleth.net/downloads/identity-provider/latest/

 cd /usr/local/src
 curl -O https://shibboleth.net/downloads/identity-provider/latest5/shibboleth-identity-provider-5.1.x.tar.gz
 curl -O https://shibboleth.net/downloads/identity-provider/latest5/shibboleth-identity-provider-5.1.x.tar.gz.sha256

Check SHA256 message digest matches:

 sha256sum -c shibboleth-identity-provider-5.1.x.tar.gz.sha256

Decompress the files

Untar the files and change to the 5.1.x directory path:

 tar xf shibboleth-identity-provider-5.1.x.tar.gz
 cd shibboleth-identity-provider-5.1.x

System Directory

Remove the system directory before upgrading to v5 (the installer will block on this issue). In v4, this directory was used for compatibility with older systems that had overridden the web.xml file, but this has been removed. In the event your v4 IdP fails to start with the system directory removed, you will need to either remove your overriden copy of web.xml, or at least freshen it by updating the very top of the file and correct the contextConfigLocation context parameter:

 <!-- Spring application context files. Files are loaded in the order they appear with subsequent files overwriting 
        same named beans in previous files. -->
    <context-param>
        <param-name>contextConfigLocation</param-name>
        <param-value>classpath*:/META-INF/net.shibboleth.idp/preconfig.xml,classpath:/net/shibboleth/idp/conf/global-system.xml,classpath*:/META-INF/net.shibboleth.idp/postconfig.xml</param-value>
    </context-param>

Further Note Regarding web.xml: There are a large number of simplifications in v5 to reduce the amount of content in web.xml and to limit the need to change it for upgrades. For this release, it may be prudent to consider rebasing your version on the delivered file. At a minimum, you should expect some warnings after upgrading to note that much of what was left behind from v4 is now deprecated or unused.

Run the installer for v5.1.x

It should detect your current source/install dir and display that on screen as below:

 ./bin/install.sh

 Source (Distribution) Directory (press <enter> to accept default): [/usr/local/src/shibboleth-identity-provider-5.1.x]

 Installation Directory: [/opt/shibboleth-idp]

 Update from version 4.3.3 to version 5.1.x
 Rebuilding /opt/shibboleth-idp/war/idp.war, Version 5.1.x
 Initial populate from /opt/shibboleth-idp/dist/webapp to /opt/shibboleth-idp/webpapp.tmp
 Overlay from /opt/shibboleth-idp/dist/plugin-webapp to /opt/shibboleth-idp/webpapp.tmp
 Overlay from /opt/shibboleth-idp/edit-webapp to /opt/shibboleth-idp/webpapp.tmp
 Creating war file /opt/shibboleth-idp/war/idp.war
 ...done

 BUILD SUCCESSFUL
 Total time: 13 seconds

Check the version has been upgraded:

 cd /opt/shibboleth-idp
 ./bin/version.sh
 5.1.x

Check plugins

Make sure that all plugins are up to date, firstly, see what plugins you have installed:

 ./bin/plugin.sh -l

For further information about IdP Plugins, please refer to this page: https://shibboleth.atlassian.net/wiki/spaces/IDP5/pages/3199500688/PluginInstallation#Overview

You may have Nashorn v1.1.0 currently installed, this is not compatible with v5.1.x and needs upgraded to Nashorn v2.0.0 to work.

Upgrade any plugins

Upgrade the Nashorn plugin to v2.0.0 if not already up-to-date:

 ./bin/plugin.sh -u net.shibboleth.idp.plugin.nashorn 

 INFO  - Including auto-located properties in ./../conf/authn/authn.properties
 INFO  - Including auto-located properties in ./../conf/saml-nameid.properties
 INFO  - Including auto-located properties in ./../conf/admin/admin.properties
 INFO  - Including auto-located properties in ./../conf/c14n/subject-c14n.properties
 INFO  - Including auto-located properties in ./../conf/ldap.properties
 INFO  - Including auto-located properties in ./../conf/services.properties
 INFO  - Downloading from HTTPResource [https://shibboleth.net/downloads/identity-provider/plugins/scripting/2.0.0/idp-plugin-nashorn-jdk-dist-2.0.0.tar.gz]  
 INFO  - Downloading from HTTPResource [https://shibboleth.net/downloads/identity-provider/plugins/scripting/2.0.0/idp-plugin-nashorn-jdk-dist-2.0.0.tar.gz.asc]
 INFO  - TrustStore does not contain signature 0xxxxxxxxxxx
 Accept this key:
 Signature:	0xxxxxxxxxxxxxxxxxx
 FingerPrint:	xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
 [yN] y
 INFO  - Installing Plugin 'net.shibboleth.idp.plugin.nashorn' version 2.0.0
 INFO  - Rebuilding /opt/shibboleth-idp/war/idp.war, Version 5.1.x
 INFO  - Initial populate from /opt/shibboleth-idp/dist/webapp to /opt/shibboleth-idp/webpapp.tmp
 INFO  - Overlay from /opt/shibboleth-idp/dist/plugin-webapp to /opt/shibboleth-idp/webpapp.tmp
 INFO  - Overlay from /opt/shibboleth-idp/edit-webapp to /opt/shibboleth-idp/webpapp.tmp
 INFO  - Creating war file /opt/shibboleth-idp/war/idp.war

Now list the plugins to see if Nashorn v2.0.0 is installed correctly:

 ./bin/plugin.sh -l
 INFO  - Including auto-located properties in ./bin/../conf/authn/authn.properties
 INFO  - Including auto-located properties in ./bin/../conf/saml-nameid.properties
 INFO  - Including auto-located properties in ./bin/../conf/admin/admin.properties
 INFO  - Including auto-located properties in ./bin/../conf/c14n/subject-c14n.properties
 INFO  - Including auto-located properties in ./bin/../conf/ldap.properties
 INFO  - Including auto-located properties in ./bin/../conf/services.properties
 Plugin: net.shibboleth.idp.plugin.nashorn	Current Version: 2.0.0

If you did not have Nashorn installed previously as you were using Java 11, you can install Nashorn like this:

 ./bin/plugin.sh -i net.shibboleth.idp.plugin.nashorn

Restart the servlet container

Since this is a major update, it's best to restart everything.

  systemctl restart tomcat10

See what attributes are being released:

 ./bin/aacli.sh --configDir=conf/ --principal=* --requester=*

You should get something similar to below showing all the attributes you're releasing and they should match what the IdP was releasing when at v4.3.3:

 {
    "name": "schacHomeOrganization",
    "values": [
        "uni.ac.uk"
    ]
  },


  {
    "name": "eduPersonTargetedID",
    "values": [
        "R45453FGEGE5GEGGSFSDF"
    ]
  }

Check the IdP process log for any errors and review the warning log for deprecation warnings and resolve:

 tail -f ./logs/idp-process.log

Check the Warn log:

 tail -f ./logs/idp-warn.log

Check the Catalina log (Tomcat):

 tail -f /opt/tomcat10/logs/catalina.out

When the IdP has started, you can exit the tail using Control-C.

Note: The v5.1.x upgrade will have kept your existing configuration files like attribute-resolver.xml, ldp.properties etc. however you will notice new similarly named files in the conf directory with .idpnew.51x on the end of the file, this is the latest v5.1.x configuration files, it's highly recommended you migrate your existing configuration settings to the same format as these files, ensuring you have a full backup of your existing configuration before doing so.

You can also test your IdP against the UK federation Test SP to ensure it's releasing attributes correctly.

Disable DEBUG

Once you are happy that the upgrade is complete and no further issues are showing in the logs:

 /opt/shiboleth-idp/conf/idp.properties

Change DEBUG to INFO in idp.properties, otherwise the logs will start taking up space with all the additional DEBUG info:

 idp.loglevel.idp=INFO
 idp.loglevel.ldap=INFO
 idp.loglevel.messages=INFO
 idp.loglevel.opensaml=INFO

IdP Troubleshooting:

Some common errors:

Has the IdP started correctly and releasing attributes?
Connection refused when restarting attribute-resolver
Normally caused by a permissions issue with the p12 certificate, check permissions on credentials directory and files.
Web page showing correct at https://your.idp.url/idp/status with no errors?
Servlet exception on web status page
You may need to add the Jakarta Servlet 5 file, see IdP browser status error below.
Any errors showing in the catalina.out (Tomcat) or idp-process.log? (Shibboleth)
Are directory permissions correct?

Tomcat Troubleshooting

IdP browser status error

If you check the IdP web page status https://your.idp.url/idp/status you may notice an error similar to below in the /opt/shibboleth-idp/logs/idp-process.log which points to a servlet exception:

 ERROR [jakarta.servlet.ServletException:144] -
 jakarta.servlet.ServletException: Handler dispatch failed: java.lang.NoClassDefFoundError: 
 jakarta/servlet/jsp/jstl/core/Config
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1104)
 Caused by: java.lang.NoClassDefFoundError: jakarta/servlet/jsp/jstl/core/Config
        at org.springframework.web.servlet.support.JstlUtils.exposeLocalizationContext(JstlUtils.java:103)
 Caused by: java.lang.ClassNotFoundException: jakarta.servlet.jsp.jstl.core.Config
        at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1353)

It will show in the browser similar to the below:

 Web Login Service - Error
 An error occurred: ServletException

This is because Jakarta Servlet 5 is looking for the Jakarta Standard Tag Library API, to correct this, change to the correct directory and download the following file:

 cd /opt/shibboleth-idp/edit-webapp/WEB-INF/lib
 wget https://repo1.maven.org/maven2/jakarta/servlet/jsp/jstl/jakarta.servlet.jsp.jstl-api/3.0.0/jakarta.servlet.jsp.jstl-api-3.0.0.jar

Rebuild the idp.war file so it contains the new jar file required for the servlet to run correctly:

 cd /opt/shibboleth-idp/bin/
 ./build.sh

The idp.war file will then be rebuilt in /opt/shibboleth-idp/war containing the newly added jar file. If your current idp.war file is located in /opt/tomcat10/webapps you can copy this new idp.war file over your existing idp.war file.

�

Windows-specific

Windows platform support

OS Support

Shibboleth IdP v5 requires a 64-bit operating system with Java 17 64-bit installed (32-bit will not work). Supported windows operating systems (from a Java standpoint) are Windows Server 2016 (64-bit, any architecture) or newer. You should be running a current supported Windows environment, search for your environment from here to see life cycle of the product

https://support.microsoft.com/en-gb/lifecycle/search?alpha=Windows

Proposed steps for upgrade from Shibboleth IdP v4:

Prepare OS:
1. Upgrade windows to 2016+
Fulfil prereqs:
1. Upgrade java to 17
2. Upgrade to latest Shib v4
Upgrade Shibboleth

Take snapshots (if VM) and/or ensure you have usable backups before carrying out any changes to your system. There will be several periods of service downtime during the upgrade process (beyond the usual ~20 seconds on unresponsiveness for a service restart).

Windows server

Ensure your OS is Windows server 2016 64-bit or newer. OS upgrades are outside the scope of this document, there are many guides as to how to do this on the internet

Java upgrade - Windows

Upgrading Java

Amazon Corretto 17 for Windows is the fully supported Java Distribution for shibboleth 5 Identity Provider. Installation instructions are available from Amazon Corretto.

In contrast, Shib v4 will run on Java 11 or 17 - the main difference in running Shib v4 on these 2 Java versions is the handling of Javascript within configs (for instance in attribute resolver). The Nashorn plugin will be required if you're upgrading from Java 11 to Java 17 as the Javascript engine is no longer included in Java 17.

Jetty upgrade for Windows

Version 5 of the IdP requires Jetty v11 or greater. The current situation for Jetty on Windows is that the latest installer uses Jetty v11, although we expect this to updated to Jetty v12 very soon, the recommendation is to use the latest Shibboleth Project provided Jetty installer.

As of January 2024 Jetty v11 has already reached "end of community support" although security updates are still being provided until it is deemed end of life, which we expect will be January 2025, going on previous lifespans.

The Shibboleth project will be releasing an updated Shibboleth IdP Jetty Configuration Baseline (jetty-base) for Jetty 12 on Windows soon. In the meantime, our advice to deployers of Jetty on Windows is that you should be running the latest provided Shibboleth Project Jetty 11 installer, and utilise this until such time as a Jetty 12 installer on Windows becomes available from the Shibboleth Project.

Upgrade to latest Shibboleth v4 (v4.3.3)

Read the entire guide first before proceeding with the upgrade!

Part of the process of upgrading to version 5 of the Shibboleth Identity Provider (IdP) should involve updating to the latest version 4 release.

Shibboleth upgrades direct from v3 to v5 are NOT supported, you must upgrade from v3 to v4 before going on to upgrade to v5. If you're upgrading from Shibboleth v3, please remember that this is a four year old Shibboleth build with many sub version changes, and after upgrading to v4 you MUST address any deprecation warnings before upgrading to v5. Once you have upgraded to v5 you will then need to check for further deprecations.

It's advisable you clone your current production IdP that you plan to upgrade so you can carry out a dummy upgrade in an isolated test environment to address any issues for when you carry out the upgrade on your production IdP.

It may also be useful to enable DEBUG for the log files, to assist with any troubleshooting you need to do, or if you need to send us log files to assist you with the upgrade.

 C:\Program Files (x86)\Shibboleth\IdP\conf\idp.properties

Change from INFO to DEBUG:

 idp.loglevel.idp=DEBUG
 idp.loglevel.ldap=DEBUG
 idp.loglevel.messages=DEBUG
 idp.loglevel.opensaml=DEBUG

Check for deprecation warnings

This is especially important if you have recently upgraded from Shibboleth v3. Remember you need to have upgraded to the latest v3 release prior to upgrading to v4, and there is no upgrade path direct from Shibboleth v3 to v5!

After each Shibboleth IdP upgrade you must check for deprecation warnings in the IdP WARN log, this will result in additional support time required from IdP Operators to address warnings before proceeding, otherwise you run the risk of hitting undocumented support issues later. Note that some of these deprecation warnings may only be visible in your servlet container�s log. Restart the Shibboleth IdP service after any changes are made to files.

Of relevance to eduGAIN organisations is warnings as shown below as this is still valid in v5 configs so can be safely ignored.

 WARN DEPRECATED xsi:type 'SAML2NameID', (file [C:\Program Files (x86)\Shibboleth\IdP\conf\attribute-resolver.xml]):
 This will be removed in the next major version of this software; replacement is (none)

See more Depreactions are here.

Check the currently installed version

Run an Administrator command prompt window (cmd.exe)

 cd "C:\Program Files (x86)\Shibboleth\IdP\bin\"
 version.bat"

Download the files

Check if your deployment includes Jetty (does a Jetty directory exist under C:\Program Files (x86)\Shibboleth?)

Download the latest v4 msi

Run the installer for Shibboleth v4.3.3

When you run the installer you will see an option to 'Run As User', it's worth mentioning at this point that if you have not done this previously using the recommended 'Run As User' as a standard user, it may be worth considering this now as by default no user will be specified and the webserver will be run with the default Windows Service account (which is usually highly privileged).'''

Note: When using the 'Run As User' option so that Jetty uses a restricted user account, add the user account to 'Logon as a Service' in the Local Security Policy > Security Setting > Local Policies > User Rights Assignments > 'Logon as a Service' or via Computer/Machine Configuration in a GPO/Azure policy. If you set a restricted user account and don't assign it to the 'Logon as a Service' prior to the install, it won't create the windows service and you'll get an error in the system log of the event viewer similar to:

"The shib_idp service was unable to log on as 'username' with the currently configured password due to the following error: Logon failure: the user has not been granted the requested logon type at this computer"

Run the file below with Administrator privileges, it should detect your current source/install Shibboleth directory:

 shibboleth-identity-provider-4.3.3-x64.msi

Note that by default it will install over the top of an existing installation. Make sure you tick the �install Jetty� option if prompted, if you have an existing Jetty install.

After the install completes, restart the Shibboleth 4 IdP service using the standard Windows tools (services.msc).

Check idp-warn.log for any deprecation warnings that need addressed. Restart the Shibboleth IdP service after any changes are made to any files.

Check attributes

Check that attributes are being released correctly after upgrading:

 aacli.bat --configDir=conf/ --principal=* --requester=*"

Upgrade to the latest Shibboleth 5

Read the entire guide first before proceeding with the upgrade!

Unlike in Shibboleth v4, the Windows Installation has changed significantly in Shibboleth v5 and is split into two parts � Shibboleth and Jetty. You need to install both of these.

Back up your current system either by taking a snapshot of your physical/virtual machine before attempting any upgrade, and ideally clone your current production IdP, that you plan to upgrade, so you can carry out a test upgrade in an isolated test environment to address any issues for when you carry out the upgrade on your production IdP.

Check idp-warn.log for any deprecation warnings that need addressed. Restart the Shibboleth IdP service after any changes are made to any files.

Download IdP install files

Note: Always check to see if there's an update available as the version below may not represent the latest version available:

Download the msi installer from https://shibboleth.net/downloads/identity-provider/latest5/

The Shibboleth Windows IdP upgrade instructions are available on the Shibboleth Project site for reference: https://shibboleth.atlassian.net/wiki/spaces/IDP5/pages/3199500925/Upgrading

You must be on Shibboleth IdP v4.3.3 before proceeding with the upgrade to v5.1.x and have addressed any deprecation warnings.

Check the currently installed version

Run an Administrator command prompt window (cmd.exe)

Change to the IdP installed directory and check the currently installed version, do not proceed if it's less that v4.3.3:

Do not proceed if it's less than v4.3.3:

 cd "C:\Program Files (x86)\Shibboleth\IdP\bin\"
  version.bat"

Check attributes

Make a note of the attributes that are currently being released, within v4.3.3, for comparison after upgrading:

 aacli.bat --configDir=conf/ --principal=* --requester=*"

Note down the output, for comparison later.

Install files

The upgrade to v5 will use your existing installation path, ensure you have a good backup before attempting any upgrade. There will be no options to select in the IdP installer other than Next/Finish. It will also remove any installed jetty-base you may have as part of the IdP install, so make sure you have a backup of your jetty-base in case it fails for any reason.

Note: After you install the IdP msi you will notice there will be no IdP service showing, this is normal and will appear after you install the Jetty msi.

Install v5.1.x using the following file with Administrator privileges:

 shibboleth-idp-5.1.x-x64.msi

It's worth mentioning at this point that if you have not installed your previous Jetty-base using the recommended 'Run As User' as a restricted user, it may be worth considering this now as by default no user will be specified and the webserver will be run with the default Windows Service account (which is usually highly privileged).

Note: When using the 'Run As User' option with a restricted user account, add the user account to 'Logon as a Service' in the Local Security Policy > Security Setting > Local Policies > User Rights Assignments > 'Logon as a Service' or via Computer/Machine Configuration in a GPO/Azure policy. If you set a restricted user account and don't assign it to the 'Logon as a Service' prior to the Jetty install, it won't create the windows service and you'll get an error in the system log of the event viewer similar to:

Install the latest Jetty msi from here: https://shibboleth.net/downloads/identity-provider/jetty-windows/

Do you want to run Jetty using a restricted 'Run As User'? complete the installer wizard.

Check plugins

At this point you will need to ensure that all plugins and modules are up to date, if you have any installed. Note: Running Java 17 requires to install the Nashorn plugin so the Javascript engine works.

Make sure that all plugins are up to date, firstly, see what plugins you have installed:

 cd "C:\Program Files (x86)\Shibboleth\IdP\bin\"

 plugin.bat -l

For further information about IdP Plugins, please refer to this page: https://shibboleth.atlassian.net/wiki/spaces/IDP5/pages/3199500688/PluginInstallation#Overview

You may have Nashorn v1.1.0 currently installed, this is not compatible with v5.1.x and needs upgraded to Nashorn v2.0.0 to work.

Upgrade/install any plugins

Upgrade the Nashorn plugin to v2.0.0 if not already up-to-date:

 plugin.bat -u net.shibboleth.idp.plugin.nashorn

or install if you did not have Nashorn installed before using Java 11:

 plugin.bat -i net.shibboleth.idp.plugin.nashorn

Now list the plugins to see if Nashorn v2.0.0 is installed correctly:

 plugin.bat -l

Check upgrade is complete

Now the upgrade should have completed, restart the Shibboleth v5 IdP service using the standard Windows tools (services.msc) and check for deprecation warnings in the WARN log file.

Check the IdP process log for any errors and review the warning log for deprecation issues and resolve:

 notepad.exe "C:\Program files (x86)\Shibboleth\Idp\logs\idp-process.log"

Check the Warn log:

 notepad.exe "C:\Program files (x86)\Shibboleth\Idp\logs\idp-warn.log"

Check the Jetty log:

 notepad.exe "C:\Program files (x86)\Shibboleth\Idp\Jetty-base\logs\jetty.log"

Check for deprecation warnings

After each Shibboleth IdP upgrade you must address any deprecation warnings that are highlighted in the IdP WARN log. Note that some of these deprecation warnings may only be visible in your servlet container�s log. Restart the Shibboleth IdP service after any changes are made to any files.

Of relevance to eduGAIN organisations is warnings regarding as shown below this is still valid in v5 configs so can be safely ignored.

 WARN DEPRECATED xsi:type 'SAML2NameID', (file [C:\Program Files (x86)\Shibboleth\IdP\conf\attribute-resolver.xml]):
 This will be removed in the next major version of this software; replacement is (none)

Check attributes

Check that attributes are being released correctly after upgrading:

 aacli.bat --configDir=conf/ --principal=* --requester=*"

After installation - Check permissions

By default the Windows Shib IdP install has no restrictive permissions in v5, it shouldn't be put into production without restrictions in place, the SetACL command can be used to constrain permissions. "An account to be given explicit read access to the distribution, and write access to the log folder. This is to allow the web server to be run by a limited permissions user (so the webserver cannot write to the machine), The default is that no user will be specified and the webserver will be run as the default Windows Service account (which is usually highly privileged)."

As highlighted here: https://shibboleth.atlassian.net/wiki/spaces/IDP5/pages/3254484996/SetACLCommand

Disable DEBUG

Once you are happy that the upgrade is complete and no further issues are showing in the logs:

 C:\Program Files (x86)\Shibboleth\IdP\conf\idp.properties

Change DEBUG to INFO in idp.properties, otherwise the logs will start taking up space with all the additional DEBUG info:

 idp.loglevel.idp=INFO
 idp.loglevel.ldap=INFO
 idp.loglevel.messages=INFO
 idp.loglevel.opensaml=INFO

�

General

This section contains those steps of the upgrade process that are not platform-dependent: which is to say, those steps that relate to the IdP itself.

SAML Proxying Issue:

If your Shibboleth IdP leverages the SAML proxying feature to pull attributes from Azure AD then your Azure AD metadata may fail to load. If you have this issue then you will need to add the following line to your conf/idp.properties

# opensaml.config.xml.unmarshall.strictMode = false

IdP deployers should use MDQ rather than metadata aggregates

The UK federation publishes metadata via its Metadata Publication Service describing all participating entities. This metadata file (ukfederation-metadata.xml) provides the information required for entities to know how to communicate with each other, and establishes a trust fabric permitting entities to verify each other�s identities. The IdP relies on the regular downloading of this metadata aggregate which is currently (Septermber 2020) over 58MB and will continue to grow with new entities added to the federation. This download is then processed locally within Java, including signature verification, which overall requires a large amount of RAM and CPU resources.

We have a new publication which use a newer protocol called Metadata Query (MDQ) to retrieve metadata. Rather than downloading the whole aggregate XML file, the metadata is looked up on demand by the IdP by querying a UK federation MDQ publication service. Thus, the required CPU and memory footprint is massively reduced.

Further details on MDQ and specifically on configuring metadata providers in the IdP

Note: that deployers who continue to use the federation metadata aggregate will eventually hit either the limits of either CPU and Memory available on the host or limits within the configuration such as Java memory assignment, resulting in a service outage or needing to size-up their environment to cope, and incurring further costs.

Deprecations:

Some deprecations you may encounter during the upgrade process.

Liberty Beans: unused SSO profile:

 WARN [DEPRECATED] - Spring bean 'Liberty.SSOS or Liberty.SSOS.MDDriven', (relying-party.xml):�
This will be removed in the next major version of this software; replacement is (none)

Resolution: Edit "conf\relying-party.xml" and remove all beans named Liberty.SSOS or Liberty.SSOS.MDDriven ? to clear this deprecation warning.

New role descriptor filter name - deprecated in v4.1:

 WARN [DEPRECATED] - xsi:type EntityRoleWhiteList, (file [C:\Program Files (x86)\Shibboleth\IdP\conf\metadata-providers.xml]): 
 This will be removed in the next major version of this software; replacement is EntityRole

Resolution: Edit conf\metadata-providers.xml and change EntityRoleWhiteList ? to EntityRole ?, this is the new role descriptor filter name. Further information: https://shibboleth.atlassian.net/wiki/spaces/IDP4/pages/1265631646/EntityRoleFilter

Basic X509 Credential Factory Bean:

 WARN [DEPRECATED] - Java class 'net.shibboleth.idp.profile.spring.factory.BasicX509CredentialFactoryBean':�
�This will be removed in the next major version of this software; replacement is Parent bean 'shibboleth.BasicX509CredentialFactoryBean'

Resolution: Edit conf\credentials.xml and replace all occurrences of:

 
class="net.shibboleth.idp.profile.spring.factory.BasicX509CredentialFactoryBean"

with:

 parent="shibboleth.BasicX509CredentialFactoryBean"

c14n / Legacy Principal Connector:

 WARN [DEPRECATED:128] - Spring bean c14n/LegacyPrincipalConnector, (c14n/subject-c14n.xml): This will be removed in the next major version of this software; replacement is <remove>

Resolution: Remove the bean reference to c14n/LegacyPrincipalConnector ? in conf/c14n/subject-c14n.xml

Relying Party Context:

 WARN [DEPRECATED:130] - Java class 'net.shibboleth.idp.profile.context.RelyingPartyContext': This will be removed in the next major version of this software; replacement is net.shibboleth.profile.context.RelyingPartyContext

Resolution: The templates for login and logout pages (views\login.vm, views\logout.vm) inherited from 3.x config refer to the RelyingPartyContext ? under a legacy name. Edit views\login.vm and views\logout.vm and in both, change:

 #set ($rpContext = $profileRequestContext.getSubcontext("net.shibboleth.idp.profile.context.RelyingPartyContext"))

 #set ($rpContext = $profileRequestContext.getSubcontext("net.shibboleth.profile.context.RelyingPartyContext"))

failfastInitialize:

 WARN [DEPRECATED:135] - XML Attribute 'failfastInitialize (on a ConnectionPool element)': This will be removed in the next major version of this software; replacement is failfastInitialize (on a DataConnector)

Resolution: In attribute-resolver.xml, make the following changes: Check the ConnectionPool ? element inside the LDAPDirectory ? DataConnector ? element - and remove any failFastInitialize attribute it might have.

Property idp.httpclient.filecaching.cacheDirectory is no longer supported:

 WARN [DEPRECATED:113] - property idp.httpclient.filecaching.cacheDirectory is no longer supported

Resolution: Just remove the idp.httpclient.filecaching.cacheDirectory property from conf\services.properties. It is unused, inherited from the 3.x generated config. Background: HttpClient ? configuration. The IdP has changed how it creates HttpClient ? instances (used for fetching data from remote locations). An IdP with 3.x config may produce this warning.

DUO Security:

 WARN [DEPRECATED:113] - property 'idp.duo.apiHost' is no longer supported
 WARN [DEPRECATED:113] - property 'idp.duo.applicationKey' is no longer supported
 WARN [DEPRECATED:113] - property 'idp.duo.integrationKey' is no longer supported
 WARN [DEPRECATED:113] - property 'idp.duo.secretKey' is no longer supported

Resolution: Comment/Remove these from conf/authn/duo.properties

idp.authn.Duo.supportedPrincipals is no longer supported:

 WARN [DEPRECATED:113] - property 'idp.authn.Duo.supportedPrincipals' is no longer supported

Resolution: Edit 'conf/authn/authn.properties' and remove/comment out the following:

 idp.authn.Duo.supportedPrincipals = \
� �saml2/http://example.org/ac/classes/mfa, \
� �saml1/http://example.org/ac/classes/mfa \

Remove legacy Duo configuration files, it is important to first complete the above changes to how property files are loaded, in case authn/duo.properties is listed in idp.properties:

Remove conf/authn/duo-authn-config.xml conf/authn/duo.properties{..idpnew*}

SAML2 NameID attribute encoders:

 WARN DEPRECATED xsi:type 'SAML2NameID', (file [/opt/shibboleth-idp/conf/attribute-resolver.xml]): This will be removed in the next major version of this software; replacement is (none)

Resolution: You do not need to remove this specific deprecation [1] and leaving this will not affect the IdP. SAML2NameID ? and SAML1NameID ? attribute encoders produce the above warnings indicating they were removed from the next major version. � In fact they are not, and (while still unsupported and undocumented) will produce a less aggressive �at risk� warning going forward. There are no explicit plans to remove them, but their use is incompatible with the vast majority of SAML software and should not be used in new deployments."

XML Element LDAP Property java.naming.referral - deprecated in v3:

 WARN [DEPRECATED:128] - XML Element 'LDAPProperty java.naming.referral', (LDAPConnector): This will be removed in the next major version of this software; replacement is (replacement depends on property)

Resolution: Remove/comment-out. This property can be replaced with the boolean attribute followReferrals on the LDAPDirectory ? element. However, this property was only needed with IdP software up to and including 2.x - and has been ignored by IdP 3.x and 4.x. Actually turning the referral following on might have undesired side-effects (IdP attempting to connect to other trees in the AD forest, potentially failing - and reporting a warning) - unless it is needed for the IdP operation, recommend leaving it off.

�