Planning Lean Gameplay Analytics
--
The Mighty Bear approach to privacy and player insights
Gameplay analytics — a necessary, but often poorly executed, staple of any successful game.
Having had some experience in Product Management with other game studios prior to my current role, I thought I knew analytics like the back of my hand. However, on joining Mighty Bear Games I quickly realised I had a lot to learn. New implementation requirements, rooted in a growing emphasis on data protection in games, are making gameplay analytics increasingly difficult to plan and execute effectively.
Our latest title, Butter Royale, is exclusive to Apple Arcade — a platform which, like Mighty Bear, takes privacy extremely seriously. Apple’s strict policies made it trickier to gather detailed insights on player behaviour, as many of the usual techniques used by game developers fall afoul of the guidelines. With these requirements in mind during Butter Royale’s development, I had to figure out a systematic framework for planning gameplay analytics. This approach needed to help us collect valuable insights while still being flexible enough to comply with Apple’s industry-leading standards for user privacy. In this article, I hope to share with you the framework we developed and how it has helped me identify the backbone of effective analytics.
A 6-step framework for planning gameplay analytics
Step 1: Understand your requirements
Before you start in earnest, it’s important to understand your environment.
Ask yourself:
Do you have any internal limitations (e.g. the tools you use, your budget)?
Are there any restrictions imposed by the platform holder?
Who will use the data?
What do you want to use the data for?
The main challenge at this stage with Butter Royale was planning around an inability to track player behaviour at an individual user-level. We had to work out the best way of drawing insights from aggregated, anonymised data from game sessions.
Step 2: Analyse your game structure
You probably have an outdated core loop lying around somewhere — time to update it! Get familiar with the different aspects of your game.
These could be:
What usually happens within one game session?
What does the first-time user experience look like?
What are the main actions players repeat over and over again?
How do players progress within your game?
Where are the opportunities for players to spend/convert?
In my case, I started working on analytics for a live game. Starting a new game meant I going through all possible flows, writing down what each one consisted of.
I consider the following to be very common game elements:
- First-Time User Experience or other forms of tutorial
- Game progression (e.g. character level progression, quest progression)
- Resource changes (e.g. currency, shop, inventory)
- User interface elements or buttons (e.g. settings)
- Feature usage (e.g. limited-time events)
Step 2 Example: Onboarding Tutorial
One fundamental feature of most games is the onboarding tutorial, as it introduces the player to the rules, controls, and gameplay.
Step 3: Define your goals
Now that you’re aware of your requirements and your game structure, we can talk about defining goals for your analytics. I want to emphasise that knowing who will be using the analytics, and exactly what they’ll be used for, is a tremendous help at this stage. For Butter Royale, the main users of the analytics would be the Product Manager (me) and the Game Designers. In general, we wanted to be able to understand:
- How do players interact with our game?
- What parts do they really enjoy?
- Which features are not used as much?
- Is our balancing working as intended? Or are there parts that our players find too difficult?
For us, it was important to be clear on how the data we want to track can be used to validate our initial projections, help us create new hypotheses, and provide actionable insights.
Ultimately, our goal was to use this information to improve engagement and the game’s overall rate of player retention.
Step 3 Example: Onboarding Tutorial
Is your tutorial engaging and accessible? How can you measure that?
One way could be to set up a tutorial funnel to be able to see which of its steps cause the most people to drop off.
Step 4: Create a tracking plan
Wooohooo! Now comes the fun part — data entry! :)
To identify the most important metrics, we created a spreadsheet that included the following columns:
- All the questions we had about our game
- Why each question matters
- Which metrics could be tracked to answer that question
- Whether those metrics would comply with Apple Arcade’s requirements
All actionable and implementable metrics were then extracted to a new spreadsheet to determine their respective importance and development complexity. (You’ve probably already had a look at our impact/effort estimation process…if not, here’s a read for you!)
Step 4 Example: Onboarding Tutorial
Break downy our tutorial into discrete steps and track how many players completed each step after starting the tutorial. This will allow you to build a funnel showing how many players completed the tutorial out of all people who started it — as well as the drop-off rate between each tutorial step.
Step 5: Plan scope and implementation
If you’ve followed these steps, you’ve probably already discussed the scope with the developers when figuring out the effort involved in tracking each metric — if not, now is the time to talk about it.
It is important to set up a structure of your data events and determine the naming conventions. For Butter Royale, we based the events on pre-defined categories — game mode, shop, etc. Ensuring consistency here will help you work with the events later and save you a lot of time!
I’d also recommend implementing a way to test the events easily. In Butter Royale, we use a debugger that shows every event fired so we can check that the implementation is working correctly, plus whether the data is available in the visualisation tool.
Step 5 Example: Onboarding Tutorial
One possible naming convention could be:
onboarding.start
onboarding.step1
onboarding.step2
onboarding.end
Step 6: Document, document, document!
I’ll keep this one short: we all recognise the need to document our processes, but unless you stay on top of this task you’ll start letting things slip. Don’t.
Everybody working with your data down the line should be able to understand what each event means and how it can be used!
Have a document ready with a definition of each event and make sure the relevant people read and understand this document before starting work with the analytics.
Step 6 Example: Onboarding Tutorial
In the documentation, clearly state what each of the tracking events does, e.g. onboarding.start = Player enters the onbaording tutorial; onbaording.step1 = Player presses button X in the onboarding tutorial.
And now you’re ready to tackle gameplay analytics like a pro!
What are some things I learned during this process?
Start small and expand later
Analytics can be used in so many different ways and the possibilities of tracking are seemingly endless — which carries with it risk of getting lost in an excess of data.
Stay focused on a small subset of metrics for your initial implementation and validate their importance, actionability, and insightfulness. That approach will minimise your investments and help you move fast.
Iterate and refine
Analytics is not something you can implement once and you’re done. As mentioned above, start small, see what works, and iterate on your plans. Not everything you thought was useful will be.
Add an analytics section to your game design document. This will help your team consider the necessary metrics for each new feature from the start and incorporate it into the development plan as early as possible.
Remember your dashboards
Looking back, it would have helped with prioritising if I had already thought about how to structure the game’s data on results dashboards. When I worked on the dashboards I noticed that some of the metrics were not as actionable as we thought they would be. This could have been prevented by setting down the structure of the different dashboards, what they would show, and in which way we could have slimmed down our data implementation.
Share your insights!
After creating dashboards, we started sharing Butter Royale’s insights in our weekly studio meeting. This helps to educate the team about the game’s performance and motivates all team members by showing them the impact their work has in tangible terms.
I hope you’re excited and ready to work on your own gameplay analytics! Feel free to check out Butter Royale, as well as the other articles in Mighty Bear’s Medium series :)
