Azure Databricks User Token Management — we can end up developing a key expiry notification & auto-rotation system

Prosenjit Chakraborty
Mar 8 · 6 min read

1. Azure Data Factory (ADF) v2 — Linked Services

Data Factory > your factory name > Connections > Select Access token

2. Azure Databricks Rest API calls

Authorization = Bearer <valid user token>

3. Using JDBC-ODBC driver



1. Azure Data Factory (ADF) v2 — Linked Services

Data Factory > your factory name > Connections > Select Azure Key Vault

2/3. Rest API calls / Using JDBC-ODBC

High level flow to retrieve Databricks user token dynamically from Azure Key Vault
expires_in = 3600
Azure Active Directory > App registrations > select the app > Settings > Keys
Key Vaults > your key vault name — Secrets > Create a secret
Azure Databricks > select the Account icon > User Settings > Access Tokens
Your Key Vault > Access policies > Add new > Add access policy > Secret permissions

User Token Refreshment

Your Key Vault > Secrets > your secret > New Version

Key / Secret / User Token Expiration

Databricks User Token Expiration:

High level flow diagram to monitor the Databricks user token expiry, create a new token and update the Azure Key Vault secret, so clients using the KV secret can use the latest token seamlessly.

Azure Key Vault Secret Expiration:

High level flow diagram to monitor the Azure KV secrets expiry and update if required, so clients can use the secrets seamlessly

Service Principal Key Expiration:


Expiry Notification


Solving the Misleading Identity Problem


Prosenjit Chakraborty

Written by

Tech enthusiast, works with Apache Spark, Scala, Kafka & Azure Services.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade