How to improve voice skill customer engagement with VoiceHero

Unique insights gleaned from looking at customer usage data

Garrett Vargas
Sep 10 · 5 min read

I recently tried an early version of a product from the Alexa Accelerator company VoiceHero. VoiceHero provides a platform that gives insights into skill usage with actionable data. I’ve used other voice analytics platforms. What sets VoiceHero’s solution apart is its ability to segment sessions and users based on custom-defined events.

I integrated two of my skills with VoiceHero during my test. With Slot Machine, I wanted to better understand patterns among people who spent a lot of time with my skill. And with Roulette Wheel, I wanted to understand if a new feature was impacting customer behavior.

VoiceHero Dashboard

Dashboard view of VoiceHero

The VoiceHero dashboard is clean and easy to navigate. They give you several ways to understand customer behavior of your skill:

  • Events — custom events that you define based on how you want to measure your skill usage
  • Cohorts — groups of users that meet certain conditions, such as triggering your events over a period of time
  • Funnel Analysis — a view of how people progress through different stages of your skill
  • Sessions and Users — a raw look into each of the sessions and users of your skill

Integrating with VoiceHero

Integrating with VoiceHero is easy. Once you’ve registered an account with VoiceHero, you receive a key to use with their service. If you’re using the Alexa SDK, you only need to wrap the result of calling addRequestHandlers with a call to the VoiceHero SDK:

const skillBuilder = new Alexa.SkillBuilders.standard();
const skillFunction = skillBuilder.addRequestHandlers(
...
)
.lambda();
if (process.env.VOICEHEROKEY) {
const voicehero = require('voicehero-sdk')(process.env.VOICEHEROKEY).alexa;
voicehero.handler(skillFunction)(event, context, (err, response) => {
callback(err, response);
});
} else {
skillFunction(event, context, (err, response) => {
callback(err, response);
});
}

A nice touch is that you don’t have to register different applications with VoiceHero. They look at the incoming skill ID and use that to set up applications in your dashboard. This makes integration a copy/paste of the above code across different skills. Currently VoiceHero only works with the Javascript Alexa SDK. But they are planning to make integrations for other implementations.

Building a cohort list

List of events in VoiceHero for Slot Machine

To understand the behavior of users who played my skill a lot, I first created an event to define heavy usage. Then I created a cohort that allowed me to do some analysis. In my case, I wanted to capture sessions that lasted more than 30 minutes as a heavy user.

Creating an event for sessions over 30 minutes

The event editor made this simple — I looked for sessions with a length of more than 1800 (30 * 60 seconds). You can also stack conditions to create events based on a combination of factors.

Once I had this event created, I was able to set up a cohort to capture all users that had a session that met this event. This allowed me to view these users, and drill into their sessions, in a single view.

Looking at heavy users of Slot Machine in the past week

Actionable Results

OK, so now I had a view into heavy users of my skill. What could I conclude from their usage patterns? Amazon had recently suggested that I integrate Alexa Presentation Layer (APL) into my skill for richer visual displays. But was it worth it? Would it help with the core set of customers that drove most of my usage, and in return, the monthly Developer Rewards I was receiving?

Using VoiceHero, I could drill down into sessions played by my “Long Session Player” cohort. For each of them, I looked at the incoming payload from Alexa to see if it was a display device or not. And there was an interesting finding. While more than a third of my total user base was using a display device, none of my heaviest users had a device with a screen. That’s quite interesting and shows me that moving to APL may be a waste of time for my most loyal customers.

Debugging new features

Roulette Wheel is a skill that I launched more than two years ago. One of its popular features is a weekly competitive tournament. I recently made this a paid feature (grandfathering in those who had played it in the past). I wanted to know how the upsell messaging was impacting users — were they leaving the skill after hearing it?

The problem was, I hadn’t defined events in VoiceHero for this, even though I had been capturing data for several weeks. Not a problem! I was able to create a new event and VoiceHero retroactively filled in sessions that matched!

All I had to do was define my events. I created an event to capture sessions with the Connections.Response request type. This indicates that they had heard a purchase prompt. I added a second condition that the session length needed to be less than 60 seconds. I compared this to all sessions with a purchase prompt to conclude that 27% of my upsells resulted in a short session. Not too bad — people weren’t leaving in droves as a result of this prompt. And I had a new customer buy their way into the tournament during the first week — nice!

Conclusion

VoiceHero is a powerful new way to gain actionable insights into your skill usage. I appreciated the way that I could easily define segments at both the session and user level to help analyze customer behavior. That said, there are a few limitations that I hope will be addressed as they open their tool for more users:

  • Events can only be defined at the intent level. I’d like to be able to specify session attribute details to help filter for usage patterns
  • Events should allow for complex query matching. In Slot Machine, I wanted to capture how many people played the game. This is not simple intent matching. A user could spin the wheel, say yes when asked if they are ready to play (e.g. in a specific game state), or press an Echo Button. I’d like to be able to define and capture this complex event to build powerful funnels
  • I would love to see a query language. For the analysis into display device usage, I’d rather enter a query for VoiceHero to execute as opposed to drilling into each session myself

I can’t wait to see how VoiceHero evolves to help me address customer pain points and deliver a richer, more engaging experience!

Garrett Vargas

Written by

CTO at CarRentals.com , part of the Expedia Group, and active voice developer and enthusiast with more than a dozen published Alexa skills.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade