Installation and configuration of Shibboleth IdP v3 software.

The Shibboleth v3 IdP has recently been released. The UK federation support team has experience of installing and configuring the software, and we are starting to develop documentation. This page therefore contains some advice, pointers to the Shibboleth project documentation, and some UK federation-specific configuration information. Please contact the UK federation helpdesk for further advice.

• IdP v3 Training
• Web server - to proxy or not
• Upgrading from v2
• New v3 deployments
• Configuration
• Registering your Shibboleth IdP
• Testing new IdP deployments
• Other information

Anyone interested in deploying the v3 IdP should start at the Shibboleth IdP v3 main page and read the system requirements and release notes.

IdP v3 Training

Jisc Training operate Shibboleth IdP and SP training courses. Please see their website for details:

Web server

It used to be necessary to proxy the Shibboleth IdP v1.3 software through the Apache web server. This is no longer necessary, and has not been since Shibboleth IdP v2.x onwards. However, for various reasons, such as easier certificate management, some deployers have preferred to use the Apache httpd web server as a proxy. For that reason the UK federation gave instructions for deploying a Shibboleth v2 IdP with either a standalone Tomcat servlet container, or Tomcat proxied through Apache httpd, Tomcat being the most popular Java servlet container for Shibboleth at that time.

The Shibboleth IdP v3 system requirements page notes that the only supported servlet containers are Tomcat 8, Jetty 9.2 and Jetty 9.3, and that the recommended servlet container is Jetty (information taken on 2016-04-21). These servlet containers can all act as standalone web servers.

It is still possible to proxy the Shibboleth IdP through Apache. There are implications for deployers who plan to do this, however:

• There are still significant numbers of SPs in the UK federation which use SAML 1, which requires a back channel AttributeQuery for secure passing of Attributes
• The Shibboleth wiki states that if you wish to proxy through Apache httpd you need to use an AJP-based proxy connector to your Java container in order to support the back channel
• The Jetty wiki states that there is no AJP support in Jetty 9

Therefore, if you do plan to proxy through Apache, and particularly if you need to support the attribute query back channel, you need to use Tomcat as the servlet container.

Upgrading from v2

It's possible to upgrade a v2 deployment to v3. It is not recommended to upgrade a production v2 deployment in place, but you could clone a production v2 deployment and upgrade that. You should make sure that you follow all the post-upgrade tasks before moving to the configuration steps below.

New v3 deployments

For a new deployment, you should first read the Installation page.

For a new Windows deployment most deployers should probably use the Windows installer, but please read the main Installation page before moving to the Windows installation page.

Linux users deploying from scratch will need to determine whether to use the Tomcat or the Jetty container and follow the appropriate links in the "Non-Windows Installation" section of the Installation page to set up their chosen container and install the software:

Configuration

General guidance

Typically the IdP installation directory is /opt/shibboleth-idp on Linux, or C:\Program Files\Shibboleth\IdP or C:\Program Files (x86)\Shibboleth\IdP on Windows. The installation directory is referred to in configuration files as %{idp.home}, and we refer to it as such here. Configuration files are located in the conf subdirectory of the IdP installation directory, that is to say %{idp.home}/conf.

Take configuration a step at a time; work on a particular configuration task, and test and modify your configuration until you have achieved the desired result. Check the idp-process.log and the container logs.

You can get more information by turning the logging level to DEBUG while you're configuring the IdP. To get details for many of the important processes in the IdP, set the following 3 parameters in %{idp.home}/conf/logback.xml to DEBUG:

    <!-- Logging level shortcuts. -->
    <variable name="idp.loglevel.idp" value="DEBUG" /> <!-- Default INFO -->
    <variable name="idp.loglevel.ldap" value="WARN" />
    <variable name="idp.loglevel.messages" value="DEBUG" /> <!-- Default INFO -->
    <variable name="idp.loglevel.encryption" value="DEBUG" /> <!-- Default INFO -->
    <variable name="idp.loglevel.opensaml" value="INFO" />
    <variable name="idp.loglevel.props" value="INFO" />

Reference documentation for logging configuration is available on the Shib wiki.

Generally we suggest the following order for configuring the IdP:

• user login, configuration usually in ldap.properties or jaas.config
• federation metadata
• attribute release in attribute-filter.xml and attribute-resolver.xml
• increase memory
• customise login page, configuration in views/login.vm, views/login-error.vm, messages/error-messages.properties
• perform any tasks required for going into production

Attribute release

For the UK federation's attribute release recommendations generally, please see section 7 of the UK federation's Technical Recommendations for Participants. There are four attributes that the UK federation regards as "core" attributes: eduPersonScopedAffiliation, eduPersonTargetedID, eduPersonPrincipalName and eduPersonEntitlement. The most commonly used of these are eduPersonScopedAffiliation and eduPersonTargetedID, so we focus on those here.

Attribute filter

The attribute filter configuration determines which attributes are released to which service providers, and is configured in the conf/attribute-filter.xml file. Most service providers in the UK federation require the attribute eduPersonScopedAffiliation with an affiliation value equivalent to full membership, eg. member, staff or student. The affiliation values released for a given user must be determined on the basis of the user's genuine affiliation with your organisation from your organisation's point of view.

In general the UK federation considers there to be very little personally identifying information carried by either eduPersonScopedAffiliation or eduPersonTargetedID so considers it safe to release them to any SP; the following configuration snippet does just that. However, your organisation should do its own risk analysis in consultation with its data protection officer rather than taking our word for it.

Note that this snippet would release these two attributes to any SP with which the IdP successfully authenticated; this might be a UK federation SP, or an SP imported from another federation via eduGAIN interfederation, or another SP for which the IdP has the metadata configured one way or another. More sophisticated attribute filters are possible; please see the Shibboleth documentation.

  <AttributeFilterPolicy id="releaseToAnyone">
     <PolicyRequirementRule xsi:type="ANY" />
     <AttributeRule attributeID="eduPersonScopedAffiliation">
         <PermitValueRule xsi:type="ANY" />
     </AttributeRule>
     <AttributeRule attributeID="eduPersonTargetedID">
         <PermitValueRule xsi:type="ANY" />
     </AttributeRule>
     <AttributeRule attributeID="eduPersonTargetedID.old">
         <PermitValueRule xsi:type="ANY" />
     </AttributeRule>
   </AttributeFilterPolicy>

Also note If your previous attribute release policies had depended on your use of a metadata aggregate tagged with groupID="http://ukfederation.org.uk" or the equivalent for metadata aggregate files from other federations (see the Benefits and risks section of the MDQ page), you may find the method for tagging per-entity metadata as described in the Internet2 Metadata Distribution Service documentation useful.

Attribute resolver

The attribute resolver configuration specifies how attributes are retrieved or generated on behalf of your users. You will need to make sure that there is a resolver:DataConnector element set up in the file to connect to your LDAP, assuming the attributes for your users are stored in an LDAP, or attributes or groups from the LDAP can be used to generate attributes. A database might also be used to store attributes.

There is an example attribute-resolver.xml file in the conf directory containing an example LDAP resolver:DataConnector configuration, called attribute-resolver-ldap.xml. The resolver:DataConnector configuration will need to be modified for your local set-up.

eduPersonScopedAffiliation

Assuming you have a resolver:DataConnector configuration with id="myLDAP", and the LDAP has been populated with the eduPersonAffiliation attribute for all your users, then you can use the eduPersonAffiliation attribute as the basis to generate eduPersonScopedAffiliation by means of the following snippet in attribute-resolver.xml:

  <resolver:AttributeDefinition id="eduPersonScopedAffiliation" xsi:type="ad:Scoped"
              scope="%{idp.scope}" sourceAttributeID="eduPersonAffiliation">
        <resolver:Dependency ref="myLDAP" />
        <resolver:AttributeEncoder xsi:type="enc:SAML1ScopedString"
              name="urn:mace:dir:attribute-def:eduPersonScopedAffiliation" encodeType="false" />
        <resolver:AttributeEncoder xsi:type="enc:SAML2ScopedString" name="urn:oid:1.3.6.1.4.1.5923.1.1.1.9"
              friendlyName="eduPersonScopedAffiliation" encodeType="false" />
    </resolver:AttributeDefinition>

If the LDAP has not been populated with eduPersonAffiliation and it is not possible to arrange for it to be populated then you may need to script an attribute definition for eduPersonAffiliation, based on other attributes or groups in the LDAP.

eduPersonTargetedID

This attribute again depends on having a resolver:DataConnector configuration to connect to your LDAP. It also requires a ComputedId resolver:DataConnector.

Please note: the ComputedId has been formally deprecated and the Shibboleth team recommend that it and the eduPersonTargetedID attribute are no longer used, and that a persistent ID should be released as a subject name identifier instead. Please see the ComputedId documentation and the subject name identifier documentation.

However, there are legacy SPs and non-Shibboleth SPs that may not be able to accept subject name identifiers in place of eduPersonTargetedID, so the UK federation recommends that UK federation IdP operators continue to release eduPersonTargetedID. Essentially the UK federation's attribute recommendations have not changed.

Assuming your resolver:DataConnector configuration has id="myLDAP" then the ComputedId resolver:DataConnector might look like this:

  <resolver:DataConnector xsi:type="dc:ComputedId"
                            id="computedID"
                            generatedAttributeID="computedID"
                            sourceAttributeID="uid"
                            salt="%{idp.computed.salt}">
        <resolver:Dependency ref="myLDAP" />
    </resolver:DataConnector>

In this example, we've used the source attribute uid which is often used in linux systems to record the users' login names. Whichever source attribute is chosen, it should not change over time.

In Active Directory, the users login name is often called sAMAccountName, although this has been known to change during the lifetime of a user's account. Depending on your system, either objectGUID or objectSid could be used. Here is an example ComputedId resolver:DataConnector that uses the objectGUID:

  <resolver:DataConnector xsi:type="dc:ComputedId"
                            id="computedID"
                            generatedAttributeID="computedID"
                            sourceAttributeID="objectGUID"
                            salt="%{idp.computed.salt}">
        <resolver:Dependency ref="myLDAP" />
    </resolver:DataConnector>

The objectGUID is binary, so if it is to be used as the source attribute it should be declared as binary by adding this line to the LDAP DataConnector with id="myLDAP":

 <dc:LDAPProperty name="java.naming.ldap.attributes.binary" value="objectGUID"/>

If you use objectSid, you'll need to add a similar element to the LDAP DataConnector.

Note that the order of the elements within the DataConnector matters. Please consult the LDAP Connector documentation on the Shibboleth wiki for the appropriate placing of the LDAPProperty.

The property idp.computed.salt can be set in idp.properties or possibly another properties file used for storing passwords, or the value can be included directly in the DataConnector. It is a random string of at least 16 bytes which you choose. It should be kept secret.

If you are migrating from an earlier version of the software you should use the same source attribute and SALT value as you used previously, which will ensure that the same values are generated for each of your users. This is necessary to retain access to personalisations saved on SPs that are keyed to eduPersonTargetedID values.

There are two forms of eduPersonTargetedID, a scoped form which we refer to here eduPersonTargetedID.old and the unscoped form which we refer to as eduPersonTargetedID. We suggest you release both forms, to allow for older SPs that do not recognise the unscoped form. Here are the attribute definitions for these two forms:

   <resolver:AttributeDefinition xsi:type="ad:SAML2NameID" id="eduPersonTargetedID"
        nameIdFormat="urn:oasis:names:tc:SAML:2.0:nameid-format:persistent" sourceAttributeID="computedID">
        <resolver:Dependency ref="computedID" />
        <resolver:AttributeEncoder xsi:type="enc:SAML1XMLObject" name="urn:oid:1.3.6.1.4.1.5923.1.1.1.10" />
        <resolver:AttributeEncoder xsi:type="enc:SAML2XMLObject" name="urn:oid:1.3.6.1.4.1.5923.1.1.1.10" friendlyName="eduPersonTargetedID" />
   </resolver:AttributeDefinition>

   <resolver:AttributeDefinition xsi:type="ad:Scoped" id="eduPersonTargetedID.old"
              scope="%{idp.scope}" sourceAttributeID="computedID">
        <resolver:Dependency ref="computedID" />
        <resolver:AttributeEncoder xsi:type="enc:SAML1ScopedString" name="urn:mace:dir:attribute-def:eduPersonTargetedID" />
   </resolver:AttributeDefinition>

Attribute Release Consent

The v3 IdP includes a flexible mechanism to allow people using your IdP to consent to the release of attributes about them. This is enabled by default for new installations and is a significant change to user interaction from the v2 IdP. See our documentation on Attribute Release Consent for details.

UK federation metadata configuration

Metadata URL and metadata signing certificate

The UK federation metadata is required for the IdP to validate UK federation service providers (SPs), and for the SPs to validate IdPs. It contains SAML metadata for all registered UK federation IdP and SP deployments. The standard production UK federation metadata is available at this URL:

To secure against compromise, the UK federation metadata is signed using the UK federation's private key, and the corresponding public key must be used to verify the signature. The public key is available at this URL in the form of a self-signed X509 certificate:

The certificate is required in the IdP configuration so that it can be used to verify the signature on the UK federation metadata. It needs to be downloaded and saved to the %{idp.home/credentials} directory.

However, as this certificate secures the entire UK Federation, you should not rely on it until you have checked its authenticity. To do this, you should verify the certificate's SHA-256 fingerprint. You can use this openssl command to find out the SHA-256 fingerprint of the certificate that you have downloaded:

 openssl x509 -sha256 -fingerprint -noout -in ukfederation.pem

To verify it you need to compare the resulting value with the correct fingerprint value, which can be obtained from the UK federation team. To guard against the possibility of this web site being compromised, you should contact them by telephone. Their phone number can be found on the UK federation helpdesk contact information page.

Configuration in metadata-providers.xml

After you have downloaded the UK federation signing certificate and verified its authenticity as described above, you configure the IdP to use the UK federation metadata by editing %{idp.home}/conf/metadata-providers.xml and adding the following MetadataProvider element. There are comments in the default file distributed with the IdP that describe where this element should be placed, and there are two example configuration elements.

 
    <MetadataProvider id="UKfederationMetadata"
                      xsi:type="FileBackedHTTPMetadataProvider"
                      backingFile="%{idp.home}/metadata/ukfederation-metadata.xml"
                      metadataURL="http://metadata.ukfederation.org.uk/ukfederation-metadata.xml">

        <MetadataFilter xsi:type="SignatureValidation" requireSignedRoot="true"
                        certificateFile="%{idp.home}/credentials/ukfederation.pem"/>
        <MetadataFilter xsi:type="RequiredValidUntil" maxValidityInterval="P30D"/>
        <MetadataFilter xsi:type="SchemaValidation"/>
        <MetadataFilter xsi:type="EntityRoleWhiteList">
            <RetainedRole>md:SPSSODescriptor</RetainedRole>
        </MetadataFilter>
    </MetadataProvider>
 

Java Memory

The Shibboleth team say that at least 1.5GB (1536MB) is required for metadata files greater than 25MB. The UK federation metadata file is approaching that size, and some federation members have reported memory issues, so the federation also recommends an allocation of at least 1.5GB Java heapspace. Use 2GB if you can, and test with your particular metadata configuration.

Tomcat on Linux

If you are deploying in the Tomcat servlet container please see under "Required Configuration Changes" on the Tomcat Shibboleth page:

https://wiki.shibboleth.net/confluence/display/IDP30/ApacheTomcat8

Jetty on Linux

If you are deploying in the Jetty servlet container please see under "Required Configuration" on the Jetty 9.2 or 9.3 (as appropriate) Shibboleth page:

https://wiki.shibboleth.net/confluence/display/IDP30/Jetty92

https://wiki.shibboleth.net/confluence/display/IDP30/Jetty93

Tomcat or Jetty on Windows

On Windows you can control the amount of Java heapspace allocated using the Windows applet for your servlet container.

If you have deployed using the Windows installer and ticked the "Install Jetty" box then you can find the applet in the C:\Program Files (x86)\Shibboleth\ProcRun or C:\Program Files (x86)\Shibboleth\ProcRun folder, depending on where the software is installed, and the applet is named shibd_idpw.exe. You need to run it as administrator. The Java heapspace can be increased by changing the "Maximum memory pool" value on the Java tab. You'll need to restart the container to pick up the change. You should restart it from the Windows Services interface, not the shibd_idpw.exe applet.

If you didn't tick "Install Jetty" and are managing your own servlet container then typically you can find the applet in the bin sub-folder of your servlet container's installation folder. In Tomcat it will be called tomcat7w.exe or tomcat8w.exe, depending on your Tomcat version.

Other configuration

Please see the Shibboleth project configuration documentation.

Registering your Shibboleth IdP

See the UK federation Shibboleth IdP registration pages.

Testing new IdP deployments

Once you have registered your IdP, you can test your IdP configuration using this UK federation test service provider:

The index page contains a number of links, which invoke different versions of the Discovery Service. The links marked as "full" have a list of all federation IdPs, including those that have been registered as hidden or invisible. If you click one of these links and select your IdP from the WAYF or DS page and successfully authenticate, you should see a list of environment variables, some of which contain the values of attributes released by the IdP; this allows you to test attribute generation and release as well as simple authentication.

WAYF links use the federation WAYF and will invoke a SAML1 session, which produces two displayed assertions – one for the authentication, one for the attributes. DS links use the federation Discovery Service and may invoke a SAML2 session, which produces a single displayed assertion.

If you are testing a Shibboleth IdP, and you have trouble authenticating or releasing attributes, then ensure your log levels are turned up to DEBUG before re-testing, and check the logs; the idp-process.log is generally the most informative. If nothing is being written to the Shibboleth logs then check the Tomcat or Jetty logs; it is advisable to keep checking the Tomcat or Jetty logs anyway during the earlier stages of the installation.

You should not attempt to gain access to any live service until you have verified, by the use of the test page noted above, that your IdP is properly configured and releasing attributes correctly.

Other information

You may also be interested in the InCommon Shibboleth training notes. InCommon is the US federation, so please note that administrative details differ between InCommon and the UK federation, and that this reference is given to provide technical details.