Authenticate with Vault by HashiCorp Using WSO2 Identity Server

Published in

Identity Unlocked

10 min readMar 26, 2019

source: https://www.quintessencelabs.com/blog/securing-vault-quantum-cybersecurity/

Authentication and authorization is a crucial part of any secret management system. Thus I would say, Identity and Access management is the cornerstone for all secret management systems. Secrets, which are literally needed to be handled by the correct person with the correct privileges, cannot go into wrong hands.

In this blog, we will integrate the Identity and Access Management from the open source WSO2 Identity Server, to one of the most popular secret management systems, the HashiCorp Vault.

What is the Vault by HashiCorp?

Imagine, during the development of your application, you have a requirement to create a database connection. This requires to store credentials of the database connection and use them whenever requires. Now, where do we keep these credentials? You can keep them in a configuration file, but with a high-security risk of losing them. How about encrypting them? Still, you need to store the master key for the decryption somewhere, which again creates the same security risk.

The solution is to put them in a vault! The application will authenticate with the vault and retrieves required credentials when needed. The vault will manage our credentials securely.

The Vault by HashiCorp is such vault implementation. It can manage secrets like passwords, tokens, certificates, API keys and any other secrets in modern computing securely.

Authentication with the Vault by HashiCorp

As you probably would have noticed by now, authentication is a crucial part of a vault. The vault must make sure that each secret it holds, is accessible only by the expected audience and no one else.

HashiCorp Vault supports a token-based authentication and authorization system. Authorization here is controlling what a user can and cannot do in the Vault. A Vault Token is assigned with a set of policies, which decides what a user having this token can do within the Vault.

In the HashiCorp’s Vault, authentication is simply the process by which a user or machine gets a Vault token.

HashiCorp Vault supports multiple pluggable authentication methods. It has the OOTB support for many of the existing auth methods including GitHub, Azure, AWS, and Google Cloud.

It also supports authenticating with a JWT. This is what we are going to use in this article, in order to authenticate with vault using the WSO2 Identity Server.

Authenticating with Vault using WSO2 Identity Server

We will use the WSO2 Identity Server as a JWT provider to authenticate with HashiCorp Vault by obtaining a JWT from the Identity Server and exchange this for a Vault Token. Let’s see how we can work on this.

I am going to use a WSO2 Identity Server 5.7 instance for this tutorial.

Configure WSO2 Identity Server

The JWT based authentication we will use, requires Vault to use WSO2 IS as an OpenId Connect provider. This allows the Vault to validate the JWT in the login request by retrieving the public keys from WSO2 IS. In addition to that, Vault requires WSO2 IS instance to have a CA-signed certificate, in order to validate connections in between.

However, OOTB WSO2 IS packs a self-signed SSL certificate as its public certificate. Therefore we need to configure this with a CA-signed certificate. For this, you can use this complete guide. From here onwards, I will assume that your Identity Server is configured with a CA-signed certificate.

Now we need to export the public certificate from WSO2 IS in PEM format. Later in this tutorial, we will provide this to the Vault. Once you get the certificate, keep it in somewhere so we can use it later.

Now the next part is a little bit troublesome due to a known issue with the WSO2 Identity Server 5.7. In WSO2 IS 5.7, OIDC issuer URL is not consistent everywhere. This means it does not have a single point to configure the OIDC issuer URL used in JWT tokes, OIDC discovery response etc. This has been fixed from WSO2 IS 5.8 onwards. But by the time I’m writing this blog, version 5.8 is not been released yet. If you want additional details on this, please refer to this GitHub issue.

Now how does this affect us? The Vault requires OIDC issuer URL value to retrieve public keys and it validates this issuer URL against the iss claim in the JWT token from the login request. Now due to the mentioned issue, configuring OIDC issuer URL will not change the iss claim in the JWT token.

Therefore, we will configure OIDC discovery endpoint of WSO2 IS to match with the OOTB iss claim in the JWT tokens.

Open <IS_Home>/repository/conf/identity/identity.xml file. Search for the configuration, <OIDCDiscoveryEPUrl>. Change this value to the following.

<OIDCDiscoveryEPUrl>${carbon.protocol}://${carbon.host}:${carbon.management.port}/oauth2/token</OIDCDiscoveryEPUrl>

We will provide this URL as the OIDC issuer to the Vault later. It will append /.well-known/openid-configuration to this issuer and retrieves OIDC discovery document from WSO2 IS. But any HTTP request with a URI containing /.well-known is secured in WSO2 IS thus requires authentication. Therefore we need to remove this constraint, to let the Vault to talk with WSO2 IS.

Open <IS_Home>/repository/conf/identity/identity.xml file. Search for the configuration <Resource context=”(.*)/.well-known(.*)” secured=”true” http-method=”all”/> under<ResourceAccessControl>. Set secured option to false. Now it should look like below.

<Resource context="(.*)/.well-known(.*)" secured="false" http-method="all"/>

Our WSO2 IS instance is now ready to talk with the Vault. Before we move on from WSO2 IS, let’s get a JWT ID Token for the Vault authentication.

But before we request for a JWT ID token, we need to configure WSO2 IS for our ID token request. First, let’s create a service provider for our JWT request.

Go to the <IS_HOME>/bin and run the wso2server.sh script to start the WSO2 IS instance.

sh wso2server.sh

Go to Home > Identity > Service Providers > Add. Enter the name as vault.jwt.provider. Click register.

Open up Inbound Authentication Configuration > OAuth/OpenID Connect Configuration and click on the Configure button.

Under the Register New Application forum, set the Callback URL to the desired value. Once a user is successfully authenticated with WSO2 IS the Relying Party(the client who made the authentication request, usually our browser) will be redirected to the callback URL with the required ID token as a query parameter.

For this tutorial, I am going to set this value to the Google home page. There is no specific reason for this. I just need some hazel-free redirection, so I can get my ID Token from the query parameters in the redirected URL. You will understand this better when we request for the JWT later.

I will also set Enable Audience Restriction and add the value https://admin.wso2.com by typing and clicking the Add button next to it. This will be used later with the Vault, to set an audience based authentication constraint.

Leave the rest as it is. Click on the Add button. Once you return to the service provider, click on the Update button.

All right. Next up is to request for a JWT ID token using our vault.jwt.provider service provider. But if I do that now, for the request I’m about to perform next, response ID Token will not contain additional user claims. This would be bad for later because the Vault JWT authentication has awesome capabilities to deal with user claims in the ID Token.

Therefore I will direct you to this tutorial from WSO2 documentation to configure some example claims with our vault.jwt.provider service provider, create and configure a user called tom, and create an OIDC request object to request WSO2 IS to add some additional claims in the response ID Token.

Before you do that, make sure you only follow from step 2 to step 5 there. And use our vault.jwt.provider service provider for them. At the end of step 5, you will be creating a signed request object. Get that and come back here.

Ok. I will assume now you have the signed request object I asked. Use the URL below in your browser, to get a JWT ID Token for the user tom you created.

https://<server-host-name>:9443/oauth2/authorize?
        response_type=id_token
        &client_id=<client-id>
        &redirect_uri=https://www.google.com/
        &scope=openid
        &state=af0ifjsldkj
        &nonce=n-0S6_WzA2Mj
        &request=<signed-request-object>

Make sure to fill each of the placeholder values in the above request.

<server-host-name>: Hostname of your WSO2 IS instance.

<client-id>: OAuth client key value for the vault.jwt.provider service provider.

<signed-request-object>: You already know what this is!

Once you do this, you will be redirected to the WSO2 IS login page. Use credentials for the user tom. If you are asked for any user claim consent as the next step, approve them.

Upon successful authentication, you will be redirected to the callback URL configured earlier. This would be the Google home page if you configured the same URL as I did.

Notice that we have a query parameter called id_token in the response URL. Value of that would be our JWT ID Token! If you decrypt this and have a look at its payload, you can see this contains the additional user claims email and country, which we requested with the request object earlier.

What we did was to get a JWT ID token with a preferred audience value and some user claims in an easier way. You can use any other method, which would also get this JWT ID token.

Now that we have the JWT ID Token and a WSO2 IS instance ready to talk with the Vault, let’s move on to the HashiCorp Vault.

Configuring Vault and then Authenticate

For this tutorial, I will start a Vault dev server. The dev server is a built-in, pre-configured server that is not very secure but useful for playing with Vault locally. This will help us to test how we can configure JWT authentication with the Vault.

You need to follow both installing the vault and starting the dev server tutorials and start a vault dev server.

First, we need to configure Vault’s OIDC discovery URL to point WSO2 IS instance as the OIDC provider. We will also include the CA-signed public certificate of the WSO2 Identity Server.

curl \
        --header "X-Vault-Token: s.xP4tTq1Yu2UCXjwGSveWYvv4" \
        --request POST \
        --data '{"oidc_discovery_url": "https://wso2.ml:9443/oauth2/token","oidc_discovery_ca_pem":"-----BEGIN CERTIFICATE-----\nMIIFRjCCBC6gAwIBAgISA8bTJ5tl2FIN9wxw/2o2a/oPMA0GCSqGSIb3DQEBCwUAMEoxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MSMwIQYDVQQDExpMZXQncyBFbmNyeXB0IEF1dGhvcml0eSBYMzAeFw0xOTAzMjAwOTI0MzZaFw0xOTA2MTgwOTI0MzZaMBIxEDAOBgNVBAMTB3dzbzIubWwwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCNvf5/xx+8pVKQ2dhmKUj3CDQmBeNNviE4iuvGMEwpd+VtViKO8wWQ6ikrhpswnhGdZDK4OcnYLZbX9p+W7TFWkzIogIkxgAOUMDzkfpFcVxan+vH24Pn3H/+3xfhvq5j6oyBiHI4KfD5M4EEElm+sqFmkGYEELXzVd4Zr7N6LKP7N2uu9vB9brOLI3c7xBYWj1VmhcsnUSm7miWnmu842lleNQnKL0UB8dGea8Vj8lHSnAl2eZqGyAEbcahnZcL5JBzeYq7/drlX9APBtx0ZlNr3qa/vWVeXZJUi4Wl2urI9aKPO4B+WKoyA4r65vc5rO811sK19nqwDiHPEfPMv7AgMBAAGjggJcMIICWDAOBgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMAwGA1UdEwEB/wQCMAAwHQYDVR0OBBYEFLIErOBYEeEmSTl9xh8z++HVr2cGMB8GA1UdIwQYMBaAFKhKamMEfd265tE5t6ZFZe/zqOyhMG8GCCsGAQUFBwEBBGMwYTAuBggrBgEFBQcwAYYiaHR0cDovL29jc3AuaW50LXgzLmxldHNlbmNyeXB0Lm9yZzAvBggrBgEFBQcwAoYjaHR0cDovL2NlcnQuaW50LXgzLmxldHNlbmNyeXB0Lm9yZy8wEgYDVR0RBAswCYIHd3NvMi5tbDBMBgNVHSAERTBDMAgGBmeBDAECATA3BgsrBgEEAYLfEwEBATAoMCYGCCsGAQUFBwIBFhpodHRwOi8vY3BzLmxldHNlbmNyeXB0Lm9yZzCCAQQGCisGAQQB1nkCBAIEgfUEgfIA8AB2AHR+2oMxrTMQkSGcziVPQnDCv/1eQiAIxjc1eeYQe8xWAAABaZqhneAAAAQDAEcwRQIgO5+uvm24J8pjaBvWnKa8EWK7AguCPjnvHmZ6ul79hzMCIQCtqKaUznH03cLH7sSe6STzzrrAmvljtAsD1H8Weng84QB2AGPy283oO8wszwtyhCdXazOkjWF3j711pjixx2hUS9iNAAABaZqhnisAAAQDAEcwRQIhAPrHwX2N8pqafeRcDft3zkVMMLH2f3hZ4G578P/IkoSqAiAucC+AiFD9tm5uoH2uxxI1j/ZxypEV10OBouoLu1c39TANBgkqhkiG9w0BAQsFAAOCAQEAN+3orkveDvATF4kZOWSCRZvJ0qfMpdl70KI9rrtgb4z5yJN+aXLhElhxanpQ5IvRqLF1Ghf4z+1oTOxRgYI3lOgMbdF7xTHC7da5mBwTsriy/c8KbclT8ES0DEkHufNKdEMso5uQGZ4gKpRWbl6wXqndaJQ4g6sFhKae7tRvyHu7KOk+EWPfPbtGJLff7WiJrJHsifmDQhRCinmhNoA/kNErkgYYJ0Onq359DOFSzRc8glddRBvf/OqjaErp3OHVGwZ9JZ194zy1dH64KbzJ/pGyWcpFzjQmuCawszHVRrEXLKf1dkZpy5U9x/ql0YsG73Oj3HAyB/CS4dKeIPzRsg==\n-----END CERTIFICATE-----"}' \
        http://127.0.0.1:8200/v1/auth/jwt/config

Replace followings in above and make the request.

X-Vault-Token: Replace this with the root token of your vault dev server.

oidc_discovery_ca_pem: Replace this with the CA-signed public certificate of the WSO2 IS. You need to include the line breakers after and before of — — -BEGIN CERTIFICATE — — and — — — -END CERTIFICATE — — — tags.

Next, we will create a Role in the Vault. A Role is created with constraints, and a login attempt is made against a role. Now the constraints created with this Role are applied to the entities attempting to log in.

A Role is be integrated with the JWT login. Therefore we can set authentication constraints in the Role, based on the JWT token.

In this example, we will restrict our authentication to the desired audience. Only a JWT having the expected audience is allowed to log in. The HashiCorp Vault has many of such constraints which can be applied with a Role. You can refer to them in the API guide.

Let’s create the Role demo. This role requires requesting JWT to have the audience value, https://admin.wso2.com and it will identify the logged in user uniquely, by the claim email. This is why we requested the ID Token to have the additional claims so that they are used in Role constraints.

curl \
        --header "X-Vault-Token: s.xP4tTq1Yu2UCXjwGSveWYvv4" \
        --request POST \
        --data '{"bound_audiences": "https://admin.wso2.com","user_claim": "email"}' \
        http://127.0.0.1:8200/v1/auth/jwt/role/demo

Replace followings in the above request and make the request.

X-Vault-Token: Replace this with the root token of your vault dev server.

All right. Now we have a Role in the Vault and a JWT from the WSO2 IS. Lets login to the HashiCorp Vault using these two.

curl \
        --request POST \
        --data '{ \
        "jwt": "eyJ4NXQiOiJOMlF6TldFNFlqazRNamMyTmpVeU9Ea3dNR05qWldKbU9Ua3lNak5rWkdSak56WXdNRGcwTXciLCJraWQiOiJOMlF6TldFNFlqazRNamMyTmpVeU9Ea3dNR05qWldKbU9Ua3lNak5rWkdSak56WXdNRGcwTXciLCJhbGciOiJSUzI1NiJ9.eyJhdWQiOlsiV3RTTVJWeGVJdThfRDNaYjhUa2VjOEVBdFJRYSIsImh0dHBzOlwvXC9hZG1pbi53c28yLmNvbSJdLCJzdWIiOiJ0b20iLCJjb3VudHJ5IjoiU3JpIExhbmthIiwiYXpwIjoiV3RTTVJWeGVJdThfRDNaYjhUa2VjOEVBdFJRYSIsImFtciI6W10sImlzcyI6Imh0dHBzOlwvXC93c28yLm1sOjk0NDNcL29hdXRoMlwvdG9rZW4iLCJleHAiOjE1NTMxMDg3MzEsImlhdCI6MTU1MzEwNTEzMSwibm9uY2UiOiJuLTBTNl9XekEyTWoiLCJlbWFpbCI6InRvbUB3c28yLmNvbSIsInNpZCI6ImI4OWYyZDIwLTdlMGUtNDJkMy05ZmM4LTllZmY3YTRkYWU2MiJ9.RucQwZ9aHlO8ls6M4iXXntH7S_lalL2ep6_q_PO0IlJvxuu_2vdNCX13KbmqKC78IokHOP_sQNNicSA43oOQziiFtLuy5cDhE9JoU2FHvUBDFfxxtPmNeKs7WxxfHB4hYME67zdzY9QuG3iUPv3PqFY9Ox0ARH1ZZgVTzgDiEbLRuMr2IWgr9kS2NS5oI2RyVGvNe8gARz82KXaJX2EY1_3zbI7DJ_xqHNUwb9qoGqWTPEbaRnXfdZ1TchukUU_atLgoNHlbWhBdubGyXIOYtzvIRTZLXg4pg8OnlXxOTr88J8u0PmzH_R7rtI_K26jNsj1KeYK3ZINgSQjmoceIVg",\
        "role": "demo" \
        }' \
        http://127.0.0.1:8200/v1/auth/jwt/login

Replace followings in above and make the request.

jwt: Replace this with the root token of your vault dev server.

You will have the following successful response. Note that there is a client_token included in the response. We have successfully exchanged our JWT token to a Vault token!

{"request_id":"ac1baa56-fe90-a5c1-7bcb-f15937ad459c","lease_id":"","renewable":false,"lease_duration":0,"data":null,"wrap_info":null,"warnings":null,"auth":{"client_token":"s.0xCc7CljMW4Sloxy0Lmw7Ykn","accessor":"PkrNAHRNvwCxWINqMeKHaGKv","policies":["default"],"token_policies":["default"],"metadata":{"role":"demo"},"lease_duration":3600,"renewable":true,"entity_id":"f74a2743-fc8e-e3e9-6fbf-6b882cb5fbf2","token_type":"service"}}

Now you can use this Vault Token in the X-Vault-Token header and create requests to the vault.

You can also try login to the Vault using this Token if you have configured the path parameter for Vault during the installation.

vault login s.0xCc7CljMW4Sloxy0Lmw7Ykn

Which gives the successful response below.

Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.Key                  Value
---                  -----
token                s.0xCc7CljMW4Sloxy0Lmw7Ykn
token_accessor       PkrNAHRNvwCxWINqMeKHaGKv
token_duration       56m53s
token_renewable      true
token_policies       ["default"]
identity_policies    []
policies             ["default"]
token_meta_role      demo

It’s important to point out that the Vault uses Policies for user authorization. During the Role creation, we could assign policies to that Role. Any Vault Token issued against this Role, will the have the Role policies attached to it.

Going into such details is out of the scope of this blog.

For such configuration and also to make full use of the Vault capabilities, please refer to the Vault documentation.

Wrap-up

In the beginning, we understood what HashiCorp Vault is and what problem it solves. In this context, we talked about how crucial authentication is to a Vault. Then we moved on to integrating HashiCorp Vault with WSO2 Identity Server using JWT authentication. We configured both entities so that the Vault can validate incoming JWT login requests by requesting public keys from WSO2 IS. Then we created a Role in the Vault and assigned JWT authentication constraints to it. After that, we were successfully able to login to the Vault using a JWT ID token provided by the WSO2 IS, and exchange that for a Vault token.