Adding telemetry to your vscode extension

The 1% rule of the internet states that the vast majority of users do not engage with the content — they use it and see it, but leave no feedback. In some cases they might give your product one try, don’t like it, and never come back. Or not even try it! This lack of feedback makes it very hard to improve your product or market it to new users.

Enter telemetry — by automatically collecting statistics and/or errors, you can get feedback without having to conduct expensive market research.

Azure application insights makes this collection very simple (and free):

npm install vscode-extension-telemetry --save

If you want a real-world example of its useage you can take a look at how I use it in AREPL-vscode.

Once it is working you should see events appear in the metrics explorer in azure:

But the beauty of application insights is not just in the logging — Microsoft offers a sophistacted query language, similar to SQL / Splunk.

distinct users, ordered by last use date.

Some other useful queries:

// heaviest users by avg time spent using ext
customEvents | 
where timestamp < now() and name=="almenon.arepl/closed" | 
summarize timeOpen=avg(todouble(customDimensions.timeSpent)) by cloud_RoleInstance | order by timeOpen

// most frequent users by number of times opened
customEvents | 
where timestamp < now() and name=="almenon.arepl/closed" | 
summarize numEvents=count(iKey) by cloud_RoleInstance | order by numEvents

You can even project your results into graphs

customEvents | where name == 'almenon.arepl/closed' | summarize count() by client_CountryOrRegion | render piechart

The analysis leads to some interesting conclusions:

1. Despite having hundreds of downloads, the actual user count is much much lower. 5 people have used it so far with one person using it twice… not great statistics. Should pick up once I market AREPL at pycon.
2. The range of users is quite geographically diverse. You don’t just get people in California or America; there’s people from canada, italy, portugal, all sorts of places. I guess that is to be expected with internet marketing — people can see your extension from countries across the world.

Once I get more people using the extension I should be able to draw more insights — like what settings they change, for example. Or how often errors occur. Or the most popular time the extension is used. Really, sky’s the limit!

Update: further anlysis of my telemetry data can be found at:

AREPL stats for June