Reverse Engineering the Azure DevOps REST API
Automating Azure DevOps Interactions
In development organizations both large and small, patterns in tool usage emerge which can be automated and controlled using various API’s. For instance, an internal tooling engineer may often need to spin up new repositories with associated build/release pipelines that follow similar architectures. Rather than hand making every single deployment pipeline, it makes sense to develop templates that can be applied repeatedly and consistently.
For Azure DevOps, this means it’s time to take a look at the REST API for defining the structures that we want to use. JSON inputs and outputs from said API give a lot of flexibility, but also have some hidden cobwebs as we are soon to find out.
JSON Headaches
We will use a release pipeline definition as an example for how to construct Azure DevOps API requests. In this example, we are deploying two Terraform plans to both production and non-production environments in a linear fashion without branch based releases.
This release pipeline appears straight-forward, but there are many moving parts to consider.
First, there are two artifacts to download which will be run during the release. The primary artifact also has an automated release trigger for any time a build successfully completes against the master branch of the parent repository.
Next, the Test_Release stage has an automated trigger after release generation accompanied by a human touch-point for approving the stage. Within this stage, there are four tasks to be completed on an agent which deploy Terraform code to a non-production cloud environment.
Finally, the Production_Release stage is similar to the Test_Release stage except that it is releasing into a production environment and is triggered by the previous stage rather than the primary artifact.
This many configuration items creates a lot of complexity when it comes to defining the release pipeline as JSON for ingestion by the Azure DevOps API. A quick look at the documentation for Release/Definition/Create shows an elaborate maze of child objects and properties to navigate.
One would perhaps think to run a GET on an existing release definition rather than constructing this JSON from scratch. However while the GET response contains a good amount of the needed JSON, it is missing some critical elements that can cause errors when used in a PUT.
{“$id”:”1",”innerException”:null,”message”:”VS402862: Artifact source should have source information. Specify a valid source and try again.”…}
This source reference error doesn’t tell us much about what is missing, and the documentation is vague as to what data belongs in the artifact source object. In addition, the GET response contains an object that is seemingly at odds with the documentation.
As a result, another method of gathering an example JSON definition is needed as described in the next section.
Grabbing the Answer
When interacting with the Azure DevOps GUI, a user is technically consuming the DevOps REST API albeit in an obfuscated manner. When a selection is made and subsequently saved during pipeline configuration, a JSON representation of the new configuration is sent out to the Azure DevOps API. This data can be captured in-flight using the trusty network logging tools included in major web browsers. To perform a capture, the network logger is opened, the DevOps UI is interacted with, and the resulting configuration is saved like so:
In the second “definitions” POST, there is a request payload that looks quite useful.
If one grabs the raw text and formats it, it becomes clear that this payload contains the release definition JSON that is needed for a POST. In fact, the artifact object that was difficult to understand from earlier now makes itself clear. It turns out that the “id” property is actually a URI and the “name” isn’t required after all.
Using New Knowledge
Armed with this example JSON, a POST payload can be constructed using any REST capable language/tool. Postman is a useful tool for testing REST API interactions as it allows for rapid iteration on payloads without needing to modify payload constructors. To test this newfound JSON release definition, grab a personal access token (PAT) with sufficient access, base64 encode it, then place it into the request header with the other required header components. Then, drop the captured JSON with any desired modifications into the request body and send the request to the appropriate API endpoint URL which in this case is:
POST https://vsrm.dev.azure.com/{organization}/{project}/_apis/release/definitions?api-version=5.0
Tools can be created around these API interactions that dynamically construct payloads depending on the use case. From this testing, I created scripts that can inject and change stages depending on how many environments a development team needs as well as any security/compliance requirements around said environments. As always, example code can be found on my Github page.
