The four ways to build your own Copilot usage & adoption reports

Disclaimer: this isn’t going to be one of those MVP posts where existing information is simply regurgitated. This is based on my experiences over the past 18 months getting under the hood with the data try to get some meaningful usage & adoption reporting for clients.

Getting an accurate and meaningful way to view how well Microsoft 365 Copilot (referred to as “Copilot” in the rest of this piece) is being used in your organisation is quite challenging — much more than it should be.

However, one could argue that’s the case with much of the Microsoft 365 platform… anything other than Exchange, SharePoint, OneDrive, Teams, Viva Engage and Office Microsoft 365 desktop apps.

If you’ve tried looking for usage data on Forms, Bookings, Planner, To Do, Loop, Viva Insights, Lists, Whiteboard, or Sway — there isn’t any. And in most of those cases, the data simply doesn’t exist.

While Microsoft does offer some reporting for Power Automate, Power Apps, Power BI — it’s not native and requires installing additional components, keeping them updated, and even then, it’s not really to the level you’d like.

And if anyone thinks that kind of usage data isn’t important…

So why should Copilot be any different?

We hear lots of stories about amazing successes with Copilot and how quickly it’s been adopted, but how do those organisations know other than some native reporting and annecdotal evidence?

“Never let the truth get in the way of a good story” — Mark Twain.

The Challenge

Microsoft does make data available for Copilot usage, and I’ve had people offer that to me as a defense whenever I’ve raised the issue. Initially, that data was only really available in the Copilot Usage Dashboard which was made available as a Power BI report, then pulled back to being just part of Viva Insights (kind of).

This also added complications around licensing, number of users, admin roles, and a few other factors. I won’t go into the specifics as anything I write down today will be invalid in 5 minutes when it changes (again).

Also, some might say that there’s Copilot usage data in Purview. Unfortunately, that’s not the kind of data you think it is, and can’t be used for what we want to achieve.

So, let’s look at the types of data we do have available as of today, what you can do with them, and some of the challenges of working with them.

The Solution(s)… kinda

I present for your consideration, four different ways of getting usage data about Copilot in your organisation:

• Microsoft 365 Usage Reports
• Audit Log
• Viva Insights
• Copilot (AI) interactions

I’ll compare all of these methods in this post, however the last two I will dive into as separate articles.

Microsoft 365 Usage Reports Audit Log Viva Insights Copilot (AI) Interactions Data provided Last date of usage in various apps Interactions with Copilot in all* apps Summarised counts of common actions, grouped by manager Interactions with Copilot in all* apps Benefits None Very rich source of data What people are using Copilot for Rich source of data Limitations — Maximum 180 days of history
- Does not show counts
- Refreshed weekly Very heavy amount of processing required for a small subset of data — Only lists the common actions
- Only shows for managers who have at least 10 direct reports
- Cannot report based on users Does not show where in Teams actions were performed

Microsoft 365 Usage Reports

The data retrieved from the Microsoft 365 Usage Reports is the same as what you see in the Microsoft 365 Admin Center: a list of users and the last date they used Copilot in the following apps:

• Copilot chat (formerly M365 Chat, formerly Bing Chat Enterprise, also known as Business Chat)
• Excel
• Loop
• OneNote
• Outlook
• PowerPoint
• Teams
• Word

Additionally, it shows who is licensed vs. actually active.

This is what you see in the Microsoft 365 Admin Center:

There’s a bit more to it, but not much.

Initially this data was only made available via the Microsoft 365 Admin Center and could be exported to a spreadsheet. Later in 2024 it was made available as an endpoint in Microsoft Graph, allowing for programmatic retrieval.

The data is only refreshed weekly, and you can only go back up to 180 days. There are no counts of activities, just the last date it was used.

I even tried to match it to last usage dates of the underlying apps so you could see compare, but this didn’t really help. Plus Loop doesn’t show up in the Microsoft 365 apps usage data.

I also brought in general Entra ID user properties so that the data could be sliced/filtered by department, manager, office location, country, job title, company name. It could even be filtered down to the individual user.

Unfortunately, I’ve had several clients tell me that only reporting on a last used date without any count of how much it was used, is practically useless. For example, if a user shows that they last used Copilot in a variety of apps in the past week, we might be inclined to think they are active and getting value from it. But what if they only did 1–2 actions and didn’t find them useful?

Here’s some screenshots of a Power BI report I created to try to make it useful:

You can find more information about the reporting endpoints here:

• getMicrosoft365CopilotUsageUserDetail
• getMicrosoft365CopilotUserCountSummary
• getMicrosoft365CopilotUserCountTrend

Needless to say, I have abandoned this approach and as such won’t be providing my workflows or Power BI report template.

Audit Log

When I first had access to Copilot in mid/late 2023, I immediately wanted usage data as the organisations I was working with had a limited number of licenses.

As the previous method wasn’t available and the Copilot Dashboard hadn’t been launched yet, I went scouring for other sources. In the Audit Log was where I found a nugget: something called a “CopilotInteraction” activity.

Using an approach by fellow MVP Leon Armston, I was able to construct a solution that would trawl the Audit Log on a nightly basis, filter it down to the CopilotInteraction activity, and write it to a table.

Because this showed me interactions for all* apps/services in Microsoft 365 where Copilot existed — I now had a count AND date.

The locations I was able to see were:

• Copilot chat (formerly M365 Chat, formerly Bing Chat Enterprise, also known as Business Chat)
• Excel
• Forms
• Loop
• OneNote
• Outlook
• Planner
• PowerPoint
• Stream
• Teams
• Whiteboard
• Word

Where it got a little murky was I also saw data for Bing and “Office” — so I wasn’t entirely sure what these were and didn’t get to the bottom of it.

* Unfortunately, I have never seen Viva Engage activity for Copilot in the Audit Log, despite the functionality having been available for quite some time now.

However, I did have an additional benefit for Teams, in that it would show if the interaction occurred in a meeting, a chat, or a channel.

Here are some examples of the report I built based on that data:

Because this method provides a count of interactions, I was able to report on who was using what and where. It also provided the ability to see who the heaviest users are, as well as who the lowest users are.

Now, it is important to note that showing low usage and identifying those users is not something that is done for negative purposes such as punishment, internal charging, or even revoking licenses. Instead, it should be seen more as an opportunity to identify people who could use coaching, or roles/areas where Copilot does not actually provide much value (which is quite possible).

While there are ways to limit access to data seen in the Audit Log by using specific endpoints in Graph, as well as a way to filter the results from using the generic Audit Log query endpoint — neither of these support retrieving only the CopilotInteraction activity.

For this reason, I won’t be sharing my workflow or Power BI report for the Audit Log method — as I do not recommend building a solution that has access to that much potentially sensitive data (i.e. everything else it would be capturing in the Audit Log). Additionally, in a large organisation with many Copilot users this would take a lot of processing power to whittle down the results.

Viva Insights (aka “Copilot Dashboard”)

Now, this particular solution is very valuable from a business perspective — but the goal posts have moved around Viva Insights licensing, user volumne requirements, permissions, and other things.

There is also a standalone version of this report available in the Viva Insights experience, as well as more enriched versions in Power BI (if you have a full Viva Insights license).

As you can see in the above screenshot, not all Microsoft 365 apps and services are listed. Additionally, if you expand the apps what you’ll see are common actions such as summarisation or generation and counts:

Unfortunately, the counts are aggregated up to the manager level, with a minimum group membership of 10 being hard-coded into the platform.

Let’s look at an example using the table from above. Let’s say that Cecil F has 10 direct reports. As we can see, there are 171 PowerPoint actions from his team. The problem is, we don’t know if one person did all 171 actions, or if others did as well!

While we don’t necessarily need to know this level of granular information in all scenarios, it is helpful to identify champions who have successfully adopted Copilot, as well as laggards who might benefit from coaching.

But because we don’t have this data, we’ll never know.

Or can we???

As I said earlier, I will dive into this topic further — because we can de-anonymise the data.

I never liked that Microsoft made this decision for customers in Viva Insights, yet gives choice in other areas around usage. So, I figured out how to identify the usage of individual users — putting the control back into the hands of the organisation as to how they want to use this data.

Here is a crude example (as I only cracked it late last year before going on holidays):

I’m still exploring my way through the data to make it more meaningful (and prettier), but I will do a separate blog post on this very soon to explain how I was able to identify individual user-level actions, and how to make it useful in a report.

Copilot (AI) Interactions

This one is relatively new, as it appears the documentation was only published near the end of 2024 (the day I went on holidays in fact).

From first impressions, this gives us close to what the Audit Log method does — but without the unnecessary non-Copilot information to wade through.

While it does give me the same Microsoft 365 apps & services as the Audit Log data, it unfortunately does not give me the same granularity for Teams usage. That being said, I’ve only gotten this method to work in the past week and haven’t looked at great detail within the data itself to see if there are ways to identify those Teams locations.

Here’s an example screenshot of the report I’m working on:

As you can see, it’s similar to the Audit Log report — because I’m working with fundamentally the same set of data.

(Also, still no Viva Engage!)

I will share more on this particular method in the coming week, as well as providing a solution you can import into your own environment to collect and report on the data.

Conclusion

There are more than one way to report on how Copilot is being used in your organisation.

Having this information is critical for making informed decisions as to where it’s being used, who is using it, and where there is opportunity to improve and grow.

Originally published at Loryan Strant, Microsoft 365 MVP.