Mau-Mau, part 2: Performance metrics

In the second article of this series on my ‘deepest’ voice game Mau-Mau, we’re taking a look at its actual performance. Is it doing well? What does doing well even mean for a voice game?

Between January and February of 2018, Mau-Mau hit 4 milestones at roughly the same time:

• It is live for 3 months, or 100 days
• It reached ‘featured’ rank #16 of the German ‘Games’ subcategory, making it visible on the category’s first overview page.
• It reached 3000 unique users in 4000 sessions
• It processed 100.000 utterances (not counting unit tests and such)

Screenshot from the day Mau-Mau hit the 100k utterances mark

How much is happening on a daily basis?

Such aggregated numbers may or may not be impressive, they are not well suited for assessing how well it does compared to both its past usage and other Skills.
Let’s check out the daily numbers:

• Unique users: 42 ± 52 (that’s not a typo: The distribution is strongly skewed to the right)
• Sessions: 43 ± 50 (usually, there are more sessions than users per day, but during the holiday season, this pattern reversed)
• Sessions per customer: 1.28 ± 0.37 (wild deviations here as well, but the median is about 1.25)
• Utterances: 1118 ± 1146 (maybe I should drop the ‘± 2×SD’ interval, this looks really weird. The lowest I had were 285, and the highest 3157)
• Utterances per session: 27 ± 17 (utterances per day and customer are similar, but can be as high as 86)
• Session length: About 110 seconds (estimated from VoiceLabs data)

How does that translate to discoverability, engagement and retention?

In my article on high-level success factors of voice games, I pointed out that your game’s usage is determined by how well you gain new users (discoverability), keep them engaged in the game (engagement), and motivate them to come back (retention).
How is Mau-Mau performing in each of these categories?

• We already covered engagement above: In average, users play for about two minutes, and interact 27 times (every four seconds) with the game
• In terms of retention, the 10.1% of the users return within the second week after their first game, and 2.9% return in he third week
• I don’t have precise numbers for discoverability, but I would guess that about 60% of the daily users (25 out of 42) are first-time users.

How good is that in comparison to other voice games?

There is one obvious and not very helpful answer: It’s on position #12 of the German Skill store’s ‘featured’ list of the ‘Game’ sub-category, so it’s not too bad. And it’s much better than my other voice games, which I’ll talk about some time later.
(Of course, this is all based on the assumption that the ‘featured’ ranking is based on a Skill’s performance, which is not completely true.)

But can we get a deeper analysis of how good Mau-Mau performs?

Let’s try! A part of what determines a Skill’s position in the ‘featured’ ranking is certainly the long-term usage trend, so a voice game can have a given ranking either because it has a solid level of usage since months, or because it has amazing usage, but it is around only shortly.
So maybe we can learn something from comparing Mau-Mau with other games from its cohort.

Mau-Mau was released in November 8th 2017, and since then, exactly 48 other voice games have been released by the time of this writing (February 10th, 2018). Since some of them aren’t around long enough to give a fair comparison, let’s define our cohort as the 2x16 games that were released before and after Mau-Mau.
Of these, 32 games, 5 currently have a better ranking — So we can assert that Mau-Mau is in the 16th percentile of its cohorts usage ranking.

One final question: Do I get paid by Amazon for Mau-Mau?

I don’t think so. I’m not sure, because I missed that I needed to fill out the tax information in the Developer Console to be eligible for the developer reward, but only did so recently.
I’ll let you know if anything happens in this regard, but in all honesty, I don’t think there will be. My reward is in learning about voice games, and getting a connection to the #VoiceFirst community and ecosystem.

Thanks for reading! What was helpful for you in this article? How are your usage patterns, compared to Mau-Mau? What else would you liked to have read about? I’m looking forward to your comments!

If you want to learn more about some user experience optimization I did on Mau-Mau, read here!