Securing your Rasa REST Channel

This tutorial is for advanced Rasa users

I have been quite active recently in the Rasa forums, and one question i noticed popped up time and again is how to ensure that the Rasa REST webhook is secured properly.

Now if you read the rasa docs carefully,

Rasa Open Source HTTP API

There is a mention that you can secure the API endpoint using JWT authentication. However this security consideration is to protect the APIs of Rasa and thus not the connectors. Rasa expects security mechanisms of the connectors are implemented by the channel itself.

There are a couple of ways to secure your Rasa REST endpoint.

But first things first, let’s settle on an IAM Provider, you could work with your favorite cloud IAM provider, but i will just work with an IAM provider i have used in the past Auth0.

Setting up an oAuth provider

Go to https://auth0.com/ and login.

Create an API in your tenant. provide the URL as http://localhost/webhooks/rest/webhook

as we are trying to protect this API. This will also create a machine to machine application. We will use the clientID and clientSecret to retrieve access tokens for our authorization

That’s it, we now have our authentication server. You have multiple options like Okta, IBM, AWS Cognito, Azure AD, Firebase Auth.

NGINX Authentication — The difficult bit

First of all, one should always expose API endpoints via an API Gateway and one of the most popular one in the market is Nginx (https://www.nginx.com/)

I will follow blindly the nginx people and their tutorial here 🙌

https://www.nginx.com/blog/validating-oauth-2-0-access-tokens-nginx/

As Rasa comes with a default Nginx configuration when you download the Rasa-X stack, you could modify the nginx configuration to support JWT based authentication and that will secure your REST channel at the level of the proxy. Note this does not change anything at the level of Rasa/RasaX but only at the side of the reverse proxy of Nginx.

In the first step,we will use the Rasa X easy install scripts and modify Nginx configuration, map it to docker-compose and secure our REST channel

Let’s take an example.

Using Docker Compose

Go to this step of the Rasa Docs

Docker Compose Installation

Download the docker-compose file by following the installation steps. Now comes the fun part!!

Let’s create an Nginx configuration, I am just going to pull the default nginx configuration from the nginx container where all the endpoints are already mapped.

For the REST channel, i am going add a custom authentication configuration

location /webhooks/rest/webhook {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Proto $scheme;
auth_request /_oauth2_token_introspection;
proxy_pass http://docker-stack/webhooks/rest/webhook;
}
location /_oauth2_token_introspection {
internal;
proxy_method      POST;
proxy_set_header  Content-Type "application/json";
proxy_set_body    '{"client_id":"$http_client_id","client_secret":"$http_client_secret","audience":"http://localhost/webhooks/rest/webhook","grant_type":"client_credentials"}';
proxy_pass        https://YOUR-TENANT.auth0.com/oauth/token;
}

So it is quite easy to explain what is going on but of course this isn’t optimized for production, please follow the above tutorial to understand about caching. Auth0 does not provide an introspection endpoint and thus here i am using the token endpoint to retrieve a token, however you can always cache that token so that you don’t need to call the token endpoint each time and creates an expires_on directive of nginx to match up with the token expiration time.

basically before the REST endpoint is called, auth_request will make a subrequest to the token endpoint of your auth0 tenant and validate whether you have passed the right credentials.

All the code is in github

Here’s the demo

Request

curl --location --request POST 'localhost/webhooks/rest/webhook' \
--header 'client_id: CLIENT_ID' \
--header 'client_secret: CLIENT_SECRET' \
--header 'Content-Type: application/json' \
--data-raw '{
"sender": "test_user",
"message": "Hi there!"
}'

Response when you provide wrong credentials.

<html>
<head>
<title>401 Authorization Required</title>
</head>
<body>
<center>
<h1>401 Authorization Required</h1>
</center>
<hr>
<center>nginx</center>
</body>

Note: I have used the Rasa-X docker-compose stack but that isn’t important. Usually it is more about the nginx configuration placed in front of the API. You could use the Rasa open source stack and put an nginx in front.

Rasa Custom Connector

Another way if you want to avoid proxy based authentication is to use Rasa OSS and create a custom connector with the following tutorial

Custom Connectors

So let’s create a connector

Once again, you will find all the code in github.

So i added a new blueprint for the new channel that would fetch an access token based on the client_id and client_secret provided by the calling application in the headers. You would always make the client do that call at their end to retrieve the access token and thus cache the same as one would do in nginx, however I would recommend working on credentials via an actual middleware which you can also add in the Sanic server.

here’s an example

Request

curl --location --request POST 'localhost/webhooks/secure_rest/webhook' \
--header 'client_id: YOUR-CLIENT-ID' \
--header 'client_secret: YOUR_CLIENT_SECRET' \
--header 'Content-Type: application/json' \
--data-raw '{
"sender": "test_user",
"message": "Hi there!"
}'

You will find all the additional configuration published on my github:

https://github.com/souvikg10/rasa-secure-rest

Note: I have only published the custom files i have overridden on top of the Rasa-X easy install docker-compose stack. You must have your own Rasa setup and get inspired from the code to implement a security mechanism.

Feel free to write to me if you have questions: souvik.ghosh@strai.be