Running a beta with an Intercom chatbot 🤖

At Userfeed we believe in running betas before we ship to all of our users. This process enables us to capture feedback early and often in an effort to reduce the risk of building the wrong things. We can get something in the hands of the most relevant users to figure out what we need to improve on before we ship to everyone. In the past, we would manually reach out to users to see if they wanted to join the beta for a given new feature. We would typically base this on users that had expressed interest in being a beta tester in the past. If they said yes, we’d manually feature flip the beta feature for their account. This worked okay early on, but as we’ve gained users, this process has shown to be both inadequate and time consuming. Using a combination of Intercom tags and Userfeed, we’ve been able to create an Intercom chatbot that engages more relevant beta testers, while completely automating the the process all together. Here’s how we did it 👉

Chatbot audience

When you’re running a beta, you want to make sure you’re getting the right types of users to test the new feature. Depending on your company, this may be users that have requested that feature in the past, users from a specific industry, users with a specific persona, users that use specific features in your app, etc. So, the first step in building a chatbot to add users to a beta is setting up the most relevant audience to receive the bot message.

In our most recent case, we wanted to keep it simple and send the bot message to:

1. Users with more that 1 session
2. Users that expressed feedback about the need for this feature (Intercom segment)

Intercom segments based on specific feedback

Creating segments in Intercom based on tags can be super helpful when creating auto messages or custom bots. The more granular you can make the segments, the more use cases you can implement. We use our own product (Userfeed) to automatically create segments for specific types of feedback (ie. a specific feature request). The segment is created via tags and anytime someone requests that feature, the tag is automatically added to the user and their corresponding conversations. In this use case, we’re targeting users who have asked for this specific feature which is now in beta.

Intercom bot audience

Chatbot paths

First we presented an initial message informing the user of the beta, describing it in detail, and asking if they’d like to join it.

We decided to go with three paths:

Yes 😍

We created an Intercom tag that would automatically turn on the beta feature for any user with this tag. Once that was implemented, we created a follow up action for this bot path, to automatically add our Intercom tag triggering the feature flip for their account! In the past, I had to manually add these tags after I had a convo with a user. With chatbots, I can completely automate this tedious task 🤩

Nope, not yet 👎

Maybe, but I have a question 💬

One thing I hate about some of the other bots I’ve interacted with is that they don’t give any open ended options, as if there could never be any other outside use cases or responses. For this bot, we created a path for those that might have more questions about the beta or how it would affect them. This way we could add an action to assign to our team for human follow up.

**Bonus tip: Video**

If you want to personalize your bot a bit more, video can go a long way. We’ve seen a huge uptick in bot engagement for any bot that includes a video.

Onwards

We often don’t get it exactly right with the initial beta, but they give us the ability to get high quality feedback from the users we’d like to get it from. Automating this process with bots gives our customers a better experience, while offering a scalable beta process as we grow our user base. Building these bots has also given us ideas about how we could further integrate Userfeed into this process to automatically capture more feedback related to a beta. Stay tuned…

If you’d like to try out Userfeed or if you want help setting up a bot like this, feel free to reach out. We’re happy to help!