Data Product Portal Integrations 1: OIDC

How to integrate Open ID Connect with the Data Product Portal

Hi everyone! I’m excited to introduce this new blog post series to the world. I hope to share many more posts and invite you all to join me on these adventures.

What is the Data Product Portal?

The Data Product Portal is an open-source tool designed to help organisations build and manage data products at scale. It’s intuitive, flexible, and aimed at making data product management straightforward and effective.

For more information, check out our announcement blogpost. It covers the “why, what and how” of the Data Product Portal and introduces key concepts. Throughout this series, I’ll refer to the Data Product Portal as “the Portal”.

The Portal is open-source! Visit our Github and leave a star if you like what you see. You can also join our Slack community to chat with us directly.

What is This Blog Post Series?

The Portal is designed for easy setup. Simply run docker compose up or helm install on your K8S cluster, and you’re ready to go. This will present you with a bare-bones version of Portal. However, the Portal’s functionality becomes truly exciting when integrated with other services, such as AWS for role management or OIDC for user authentication.

This blog series will guide you through various integrations for the Portal with technical examples. Topics include integrating the Portal with OIDC, linking the Portal to AWS, and connecting to Conveyor.

Following this series will help you fully utilize the Portal by integrating it with essential technologies, creating a comprehensive hub for your data professionals.

This blogpost focuses on integration with OIDC.

Why Do You Need OIDC Integration?

The Portal allows large-scale data access within your organization. A crucial feature is the ability to authenticate and authorize users. By verifying user identities, the Portal can securely and correctly grant access to the appropriate data products.

Without OIDC, all users will appear as “John Doe”, making it impossible to differentiate access rights. Enabling OIDC is the first step to making your Portal installation more useful for your entire organization.

What is OIDC?

OpenID Connect (OIDC) is an identity authentication protocol that is an extension of open authorization (OAuth) 2.0 to standardize the process for authenticating and authorizing users when they sign in to access digital services.

Implementing OIDC with the Portal allows you to link the Portal to your organization’s SSO or identity provider, enabling secure user sign-in and user-specific data access.

Secure user sign-in with AWS Cognito

With disabled OIDC, everybody logs in as John Doe automatically

How to Integrate OIDC with the Portal

Supported Providers

The Portal has been tested with AWS Cognito but should work with other OIDC providers with minimal code changes. As an open-source project, contributions to support additional OIDC providers are welcome.

OIDC returns identity , refresh and access tokens. Providers may vary in the information they pass in tokens, and the application depends on this information, so integrating a new provider might require some code changes there as well.

The claims our provider is required to pass are at least email, family_name and name.

Cognito setup

While I won’t cover the complete setup of a Cognito user pool (you can find that here), here are the key points for the Portal:

• Add email, family_name and name to the required_attributesin your user pool.
• Create an app integration and note the client ID and secret.
• Find the authority URL using the format:

https://cognito-idp.<COGNITO_REGION>.amazonaws.com/<COGNITO_USER_POOL_ID>

• Set the correct callback URLs

I will dive deeper into these steps below.

Portal makes use of Cognito User Pools. When creating your user pool, make sure to add at least email, family_name and name to the required_attributes. These claims are used in portal.

Your user pool will need exactly one app integration.

Go to the app integration tab to create a new integration

We need exactly one app integration

When you click on this app integration you can find out the client ID and secret of your app client. These values are important below.

Opening the app integration shows you the Client ID and secret

The last value you need to find is the authority url. In case of Cognito you can create it with the following template.

https://cognito-idp.<COGNITO_REGION>.amazonaws.com/<COGNITO_USER_POOL_ID>
# example
https://cognito-idp.us-east-1.amazonaws.com/us-east-1_AB12ZQfCb

For other OIDC providers we require the URL that hosts the .well-known/openid-configuration file of your provider.

Callback URLs

The Portal requires specific callback URLs allowed by Cognito. These are necessary for both local development and production deployment. Ensure the following URLs are in your allowed sign-out callback list:

For local development:

http://localhost:3000/
http://localhost:3000/logout/
http://localhost:5050/
http://localhost:5050/api/auth/device/callback/

For production (HOST is the CNAME of your application — the DNS record that maps the alias name of your application to it’s true or canonical domain name):

https://<HOST>/
https://<HOST>/api/auth/device/callback/
https://<HOST>/logout/

The following list needs to be added to the allowed sign-out callback list.

http://localhost:3000/logout/ # For local development
https://<HOST>/logout/

Integrating OIDC Locally / Docker Integration

Backend

When running the portal locally, most of the integration of external services happens in a .env configuration file.
Running docker compose up uses .env.docker by default. If you want to build your dockers locally for deploying, you need to create a .env file in the root of the project. Please refer to the README in backend/ for further explanation.

Make sure the following configuration values are present in the .env file used in your Docker deployment pipeline. The values can be derived from your Cognito user pool as explained above.

OIDC_CLIENT_ID=<ID> # Cognito Client ID
OIDC_CLIENT_SECRET=<SECRET> # Cognito client secret
OIDC_AUTHORITY=<AUTH_URL>
OIDC_REDIRECT_URI=http://localhost:3000/,http://localhost:8080/ # For local dev
OIDC_DISABLED=false

Frontend

The frontend docker used when running locally or when executing npm dev uses a config.js file. Please refer to the README in frontend/ for further explanation on how to set this up. The file should contain the following code:

const config = (() => {
    return {
        API_BASE_URL: "http://localhost:5050",
        OIDC_DISABLED: false,
        OIDC_CLIENT_ID: "<CLIENT_ID>",
        OIDC_CLIENT_SECRET: "<CLIENT_SECRET>",
        OIDC_AUTHORITY: "<AUTH_URL>",
        OIDC_REDIRECT_URI: "http://localhost:3000/",
        OIDC_POST_LOGOUT_REDIRECT_URI: "http://localhost:3000/"
    }
})();

Integrating OIDC in Your Production Installation

The easiest way to deploy portal in a production environment is through Helm. You can enable OIDC in Helm by passing the following configuration in the values.yaml file provided through Helm.

Make sure your values.yaml contains the following entries.

oidc:
  disabled: false
  client_id: <client_id>
  client_secret: <client_secret>
  authority: <auth_url>
  redirect_uri: <host> #including trailing /

Conclusions

Setting up OIDC support for your Portal deployment is straightforward. Adjust the .env and config.local.js files for local development, or pass values in values.yaml for Helm installs.

Let me know if this tutorial was clear and feel free to reach out with questions. Join us on Github or in our Slack community. Stay tuned for the next installment of this series!