Can You Trigger and Track One ADF Pipeline From Another? Find Out How!

Explained End-to-end solution with source code

Typically, we initiate Azure Data Factory (ADF) pipelines through the ADF scheduler or by running them from one pipeline within the same ADF. However, there are specific situations where we may need to execute an ADF pipeline from a different ADF. This blog post is designed to guide you through the process of running the ADF 02 pipeline (referred to as the child ADF) from ADF 01 (referred to as the parent ADF) and simultaneously monitoring the status of the invoked pipeline. During the pipeline execution, we also transmit the Run ID from the parent ADF to the child ADF.

Photo by Mike Benna on Unsplash

Pre-requisite

There is only one pre-requisite. To run the ADF pipeline from another ADF, we need to provide access to ADF 01 (calling/parent ADF) on ADF 02 (called/child ADF):

• The minimum role needed is Data Factory Contributor, but you could also use a regular Contributor or Owner.
• Go to called/child ADF (ADF 02)
• In the left menu click on Access Control (IAM)
• Click on the +Add button and choose Add role assignment
• Select Data Factory Contributor as a Role