Telemetry driven releases with Application Insights and VSTS (part 2)
Using telemetry to control releases from staging to production
In part 1 we looked at how to gather telemetry from both a staging and production slot using Application Insights and how to compare the telemetry using Application Insights Analytics. Now we will take this telemetry and use it in VSTS to setup an automated release pipeline that will check for side effects before allowing changes to move into production.
We are again looking at this scenario:
What we want to achieve is deploy to a staging slot, allow some traffic to flow through the staging slot, wait 15 minutes to allow the new deployment to build up some telemetry, use a release gate to compare page load times against the same page on production and then decide if we want the release to go through.
Prepare the query
We are going to use the Application Insights API to run a query so before we can do that we need to format the query we created in part 1 so we can use it in the REST call.
The easiest way is to use the Application Insights API explorer to test and format your query. This way you can make changes to your query, check the results and once you are happy with it copy the correctly formated URL for the REST call.
We will use this query:
let AvgDuration = toscalar(requests | where timestamp > ago(24h) and customDimensions.SlotName == "production" and name == "GET Home/Index" | summarize percentile(duration, 95));requests |where timestamp > ago(5m) and customDimensions.SlotName == "staging" and name == "GET Home/Index" and duration > AvgDuration * 1.10 | summarize(count())
Past it in the “Query” field and enter the “Application ID” and “API key” for your Application Insights instance. Click “Fetch!” and you should get a result similar to the one below.
Once you are happy with the query and the results it returns go to the CURL tab. Copy the URL that is between the double quotes (“) and keep it for later.
Create the release gate
Before you can send a call to the Application Insights API you will have to configure a generic service endpoint that points to Application Insights. You can leave username and password empty since we will provide that as part of the REST call in the release gate.
We are using a very simple setup with only a staging and production environment in our production pipeline. First we will have to configure a gate by clicking on the ellipsis on the left side of the production environment box.
Enable “Gates” and expand the “Gates” section. Next add a new gate and choose the “Inovoke REST API” gate so we can invoke the Application Insights REST API to run a query.
Select the “application-insights” as the enpoint, select “GET” as the method and add the headers as shown in the example below. Make sure you have a variable called “ApiKey” that contains the API key for the application insights instance you are sending the query to.
The “Success criteria” you see in the screenshot above determine how the release gate will decide if call to Application Insights returns the correct result. These release gate conditions are a bit complex, Jesse Houwing has a nice blog that explains how to craft complex release gate conditions and he also created a tool to easily test and verify your conditions locally.
The JSON path expression used here means that we expect 0 results to come back from our query:
Our query is doing the following: take the 95th percentile of the duration of requests for the home page in production over the last 24 hours and check if there are any requests in the last 15 minutes for the homepage of staging that are more than 10% slower. When this is not the case the query will return 0. Note that if there are no requests to staging in the past 15 minutes the query will return 0 as well, so in a production scenario you will want to craft a bit more elaborate query that will also fail if there are no or not enough requests in the time frame you are checking.
That’s it, run the release and you will see your gate is triggered after deployment to staging is completed.
