Calling Azure Rest API in Data factory with Access Token
An API is a set of definitions and protocols for building and integrating application software. It’s sometimes referred to as a contract between an information provider and an information user — establishing the content required from the consumer (the call) and the content required by the producer (the response).
Representational State Transfer (REST) APIs are service endpoints that support sets of HTTP operations (methods), which provide create, retrieve, update, or delete access to the service’s resources.
Following four HTTP methods are commonly used in REST based architecture.
GET − Provides a read only access to a resource.
POST − Used to create a new resource.
DELETE − Used to remove a resource.
PUT − Used to update a existing resource or create a new resource.
OATH TOTP (Time-based One Time Password) is an open standard that specifies how one-time password (OTP) codes are generated.
In REST, the URI is path to the Resources. Resource parameter depicts the identifier of the Web API that your client wants to access on behalf of the user. Most flows in OAuth involve 4 parties, the resource owner (user), the client (app), the authority (identity provider) and the resource (web api). The audience of the access token that the authority generates is the resource identifier.
Microsoft has a very nice documentation about OAuth 2.0 client credentials flow:
https://docs.microsoft.com/en-us/azure/active-directory/develop/v2-oauth2-auth-code-flow
In the case of Azure AD you can either use the Client ID or the App ID URI of the resource Web API. For instance, if I want my client to get a token to access the Azure AD Graph API on behalf of the user, I would request for a token for resource “https://graph.windows.net".
In this post, we’re going to do a REST call to get a list of all secrets in Key Vault. This demo is strictly to show how to call API with access token. With Managed Identity authentication, the same task can be done in one step. But for the purpose of the demo, we are doing it in two steps.
- Register App in Azure Active Directory
2. For the app registered, create a secret and copy it to notepad.
3. For the app registered, copy clientid & tenantid.
4. create a Key Vault and add secrets to it. Also, copy the the Key Vault URI.
5. Create a pipeline in Data Factory and add two web activities to it. First activity gets the access token using REST API and the second activity actually calls the Key Vault REST API with the token.
6. Enter the URI value in the following format for Access_Token activity. Replace <tenantid> with the actual values from step 3.
https://login.microsoftonline.com/<tenantid>/oauth2/token
7. Add Header to the Access_Token Activity.
Name: Content-Type
Value: application/x-www-form-urlencoded
Body: @concat(‘grant_type=client_credentials&resource=https://vault.azure.net&client_id=<clientid>&client_secret=',encodeUriComponent('<client secret>'))
Replace <clientid> and <client secret> from step 2 and step 3.
8. For the second activity (Call KV with Token), enter the following value for URL. This is the REST API Call the Key Value API to get the list of secrets with maxresults = 1 (return 1 value)
https://<Key Vault>//secrets?maxresults=1&api-version=7.2
Replace <Key Vault> with the value from step-4
9. Add following Header to the activity.
Name: Authorization
Value : @concat(string(activity(‘Access_Token’).output.token_type),’ ‘,string(activity(‘Access_Token’).output.access_token))
10. Run the pipeline and check the output of each activity. The output of the first activity looks like this and the second activity fails.
The error message clearly says that the app doesn’t have the permission to access the Key Vault.
11. Now let’s go back to the Key Vault and add the app to the Access Policies. Make sure to pick List & Read permission for the Secrets
12. Run the pipeline again and check the output of the second activity. You will notice that the output now lists the first secret from the Key Vault.
If you would like us to evaluate and review your current progress with your Data/API Architecture, please email us at info@datasurge.com or complete the form on our contact us page.
