Connecting Shibboleth and Nginx

At the Cornell College of Veterinary Medicine, we have been using CuWebAuth to do authentication and authorization. As described in https://medium.com/cuc4/using-json-web-tokens-with-cuwebauth-79aab6c3396d, we prodded Apache into generating a JWT (JSON Web Token) that asserts “This user is n, and they are in roles x1, x2, x3.” This has been tremendously useful in a Microservice setup.

But We Must Move On…

Unfortunately, recent upgrades to our preferred front end framework Angular have caused problems. We’ve traced the problem back to front controller patterns. In Single Page Applications (SPA), there are only a handful of files stored on the server, with all internal routing performed by the framework. So you might hit:

• https://app.vet.cornell.edu/bulletin-board

(Not a real URL) You get the bulletin-board/index.html file and some JavaScript files from the server. You click on a post and might go to:

• https://app.vet.cornell.edu/bulletin-board/post/2

It looks like you’re hitting the app server again, but you’re not. The browser and framework are handling this all internally — redrawing content for that post and changing the URL, but essentially working with the same index.html and JavaScript files you loaded before.

So far so good. The problem is when you bookmark the post page above. If you try to get back to it again, it issues the /post/2 URL back to the server, which doesn’t actually exist. SPA frameworks like Angular and React recommend patching your server with rewrite rules that essentially pass back index.html plus the JavaScript files anyway, then return control to the browser.

In our Apache, CUWebAuth and JWT setup here’s what would happen:

• User issues https://app.vet.cornell.edu/bulletin-board-post/2
• CUWebAuth finds the request unauthenticated, and go through its authentication process
• The authentication process completes, and the rewrite rules do an internal redirect.
• The JWT issuance process would lock up the server thread for some unknown reason. The user would never get a response.

We tried moving the JWT issuer to different hooks in the Apache content filtering process, to no avail. Either the JWT would be invoked too earlier, and not get the CUWebAuth credentials … or it’d be too late and lock up the thread.

On to Nginx and Shibboleth

CUWebAuth is a fairly old monolithic Apache plugin. It runs in the same process space as Apache, and must be recompiled every time Apache is upgraded. This makes it fast, but virtually undebuggable for those who don’t know Apache and CUWebAuth internals.

We’d rather be experts at building Vet School apps than building Apache. After all, the Apache web server is slowly being abandoned for faster, more configurable, and more flexible alternatives — like Nginx. And like other IT shops, we’ve had very good luck using Nginx already. It is the underpinning of AWS Load Balancers and Kubernetes Ingress Controllers that we use all over the Vet School. We are not experts in Nginx, but we don’t need to be — we’ve been able to make it do what we want through very simple reconfiguration.

CUWebAuth won’t run on Nginx but Shibboleth will, so we figured … if we’re gonna need to spend umpteen hours getting JWT’s to work right, we might as build the infrastructure we want rather than the one that worked years ago.

In a future post, I’ll go over how we did the JWT part. But because there’s a dearth of information on running Nginx and Shibboleth together, … or rather, there’s information, but it’s very fragmented … I am going to explain that part first.

The Components

As I explained in the previous post, modern IT solutions are best when gluing small, well-debugged open sources pieces together with monitorable connectors. Our goal is to get Nginx up and running with Shibboleth authentication, and for that we’ll use at the core:

• Ubuntu Eoan (19.10)
• Ngnix 1.17
• Shibboleth SP 3 — The most important part of which is shibd, a daemon that handles the AuthN/AuthZ handshake

The problem is nginx and shibd run as two totally separate processes. So we need some extra “glue”:

• Shibauthorizer and shibresponder — two FastCGI programs provided in the Shibboleth utilities package shibboleth-sp-utilities
• Supervisord — runs the two FastCGI programs as daemons with sockets. That way Nginx can talk to them
• Nginx Shibboleth Module —a very small Nginx module that talks to Shibauthorizer and Shibresponder through sockets

All of these pieces are small and open source. Because communication is through sockets, you can in theory run shibd and nginx on separate machines, but we like to keep it simple and secure.

Installation

To install Nginx, you mostly follow the directions in http://nginx.org/en/linux_packages.html

$ sudo apt install curl gnupg2 ca-certificates lsb-release
$ echo "deb http://nginx.org/packages/mainline/ubuntu eoan nginx" \    | sudo tee /etc/apt/sources.list.d/nginx.list
$ curl -fsSL https://nginx.org/keys/nginx_signing.key | sudo apt-key add -    
$ sudo apt-get install nginx shibboleth-sp-utils supervisor

The Nginx Shibboleth module doesn’t come prepackaged, but the pkg-oss infrastructure makes it fairly easy to build. The pkg-oss directions are found in https://www.nginx.com/blog/creating-installable-packages-dynamic-modules/ and the specific instructions for nginx-shib are in https://github.com/nginx-shib/nginx-http-shibboleth

$ wget https://hg.nginx.org/pkg-oss/raw-file/default/build_module.sh
$ chmod a+x build_module.sh
$ ./build_module.sh https://github.com/nginx-shib/nginx-http-shibboleth.git

You should now have a debian package with everything you need in the file debuild/nginx-module-shibboleth_1.17.10-1~eoan_amd64.deb You can install it with:

$ sudo apt install debuild/nginx-module-shibboleth_1.17.10-1~eoan_amd64.deb

Configuring Nginx

At Cornell, you’re required to do any Shibboleth-backed apps on an HTTPS server, and that’s good practice for any server these days. We use InCommon, so if you don’t already have an SSL cert, you can do:

$ openssl req -nodes -newkey rsa:2048 -keyout $HOSTNAME.key -out server.csr

Follow the prompts as always. Then go to https://ssl.cit.cornell.edu/sslReq/request.html and paste the contents of the CSR file and wait for your certificate to arrive.

1. Move the key file $HOSTNAME.key to the/etc/ssl/ssl.key directory. Secure it 0600 as you normally would.
2. Copy the certificate to /etc/ssl/ssl.crt/$HOSTNAME_cert.cer
3. Copy the chain (certificate bundle) to /etc/ssl/ssl.crt/$HOSTNAME_interm.cer

These directory locations are de facto standards, but as long as they match what’s in nginx.conf, they should work. Speaking of which, you should edit the /etc/nginx/nginx.conf to look something like this:

user  nginx;
worker_processes  1;
# Add this line
load_module modules/ngx_http_shibboleth_module.so;
error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;
# Everything else as before...

And create a file called /etc/nginx/conf.d/ssl.conf that looks something like this:

server {
  listen       443 ssl;
  server_name  $HOSTNAME;
  access_log /var/log/nginx/ssl.log main;
  ssl_certificate /etc/ssl/ssl.crt/$HOSTNAME_cert.cer;
  ssl_certificate_key /etc/ssl/ssl.key/$HOSTNAME.key;
  location = /shibauthorizer {
    internal;
    include fastcgi_params;
    fastcgi_pass unix:/var/run/shibboleth/shibauthorizer.sock;
  }
  location /Shibboleth.sso {
    include fastcgi_params;
    fastcgi_pass unix:/var/run/shibboleth/shibresponder.sock;
  }
  location /shibboleth-sp {
    alias /etc/shibboleth/;
  }
  location / {
    root /usr/share/nginx/html;
    index index.html;
    shib_request /shibauthorizer;
    shib_request_set $shib_remote_user $upstream_http_variable_uid;
    # For optional groups attribute
    shib_request_set $shib_remote_groups $upstream_http_variable_groups;
  }
}

This essentially locks down the entire server for Shibboleth authentication. If you want to lock down some directories but not others, you can use multiple location blocks.

Configure Shibboleth

Now you need to point Shibboleth in the right direction. At Cornell, you can point your SP at the test IdP first. Later on down the line, you can switch this to the production IdP.

The first thing you should do is generate signing and encryption keys for SAML assertions. This keeps all the transactions between your server and the Shibboleth IdP nice and secure.

$ shib-keygen -u _shibd -h $HOSTNAME -e https://$HOSTNAME/shibboleth -n sp-signing
$ shib-keygen -u _shibd -h $HOSTNAME -e https://$HOSTNAME/shibboleth -n sp-encrypt

Our /etc/shibboleth/shibboleth2.xmlfile points at test IdP:

<SPConfig xmlns="urn:mace:shibboleth:3.0:native:sp:config" xmlns:conf="urn:mace:shibboleth:3.0:native:sp:config" clockSkew="180">
<OutOfProcess tranLogFormat="%u|%s|%IDP|%i|%ac|%t|%attr|%n|%b|%E|%S|%SS|%L|%UA|%a" />
<RequestMapper type="XML">
  <RequestMap>
    <Host name="$HOSTNAME" authType="shibboleth" requireSession="true">
    <Path name="authz_example">
      <AccessControl>
        <Rule require="groups">ExampleGroup</Rule>
      </AccessControl>
    </Path>
  </RequestMap>
</RequestMapper>
<ApplicationDefaults entityID="https://{{ inventory_hostname }}/shibboleth" REMOTE_USER="uid eduPersonPrincipalName" cipherSuites="DEFAULT:!EXP:!LOW:!aNULL:!eNULL:!DES:!IDEA:!SEED:!RC4:!3DES:!kRSA:!SSLv2:!SSLv3:!TLSv1:!TLSv1.1">
  <Sessions lifetime="28800" timeout="3600" relayState="ss:mem" checkAddress="false" handlerSSL="true" cookieProps="https">
    <SSO entityID="https://shibidp-test.cit.cornell.edu/idp/shibboleth">
    </SSO>
   <Handler type="MetadataGenerator" Location="/Metadata" signing="false" />
   <Handler type="Status" Location="/Status" acl="127.0.0.1 ::1" />
   <Handler type="Session" Location="/Session" showAttributeValues="true" />
  </Sessions>
  <Errors supportContact="YOUR_NETID@cornell.edu" helpLocation="/about.html" styleSheet="/shibboleth-sp/main.css" />
  <MetadataProvider type="XML" validate="true" url="https://shibidp-test.cit.cornell.edu/idp/shibboleth" backingFilePath="cornell-idp-test.xml" maxRefreshDelay="7200" />
  <AttributeExtractor type="XML" validate="true" reloadChanges="false" path="attribute-map.xml" />
  <AttributeFilter type="XML" validate="true" path="attribute-policy.xml" />
  <CredentialResolver type="File" use="signing" key="sp-signing-key.pem" certificate="sp-signing-cert.pem" />
  <CredentialResolver type="File" use="encryption" key="sp-encrypt-key.pem" certificate="sp-encrypt-cert.pem" />
</ApplicationDefaults>
<SecurityPolicyProvider type="XML" validate="true" path="security-policy.xml" />
<ProtocolProvider type="XML" validate="true" reloadChanges="false" path="protocols.xml" />
</SPConfig>

In this case, a valid Netid must be used for any file on the site, and anything under authz_example must be a member of ExampleGroup.

Connecting Nginx to Shibboleth through Supervisord

If you have restarted Nginx at this point, it’ll probably error because the sockets /var/run/shibboleth/shibauthorizer.sock and shibresponder.sock have not been created yet. That’s the job of Supervisord, and it mediates communication between these two entities.

First you create the service definition for Supervisor itself in /etc/supervisor/supervisord.conf

; supervisor config file
[unix_http_server]
file=/var/run/supervisor.sock   
chmod=0700                       
username = dummy
password = dummy
[supervisord]
logfile=/var/log/supervisor/supervisord.log 
pidfile=/var/run/supervisord.pid 
childlogdir=/var/log/supervisor 
user = root
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
[supervisorctl]
serverurl=unix:///var/run/supervisor.sock 
[include]
files = /etc/supervisor/conf.d/*.conf

Then you create interfaces for the two Shibboleth Fast CGI programs. These go in /etc/supervisor/conf.d/shibboleth.conf

[fcgi-program:shibauthorizer]
command=/usr/lib/x86_64-linux-gnu/shibboleth/shibauthorizer
socket=unix:///var/run/shibboleth/shibauthorizer.sock
socket_owner=_shibd:_shibd
socket_mode=0666
user=_shibd
stdout_logfile=/var/log/supervisor/shibauthorizer.log
stderr_logfile=/var/log/supervisor/shibauthorizer.error.log
[fcgi-program:shibresponder]
command=/usr/lib/x86_64-linux-gnu/shibboleth/shibresponder
socket=unix:///var/run/shibboleth/shibresponder.sock
socket_owner=_shibd:_shibd
socket_mode=0666
user=_shibd
stdout_logfile=/var/log/supervisor/shibresponder.log
stderr_logfile=/var/log/supervisor/shibresponder.error.log

Logging in both the responsder and authroizer, and in Supervisord itself, gives you a fighting chance in debugging most problems. Restart everything:

$ sudo service restart shibd
$ sudo service restart supervisor
$ sudo service restart nginx

And you should have a working Shibboleth-locked web server. Put your content in /usr/share/nginx/html and away you go!

Next

With these pieces in place, we’ll do JWT’s in a later post. Much like the CUWebAuth solution where we used mod_ruby, we can plug in a high-level programming language to do what we want with very few lines of code. In Nginx’s case, we’ll use njs which is a flavor of JavaScript.