Heap — A Simple Way to Understand Your Product

Overview

You’ve just launched a new product and are eager to see how it’s doing. You want to iterate, but you’re not entirely sure which features to improve just yet. You also may not be extremely comfortable with tools like Tableau or know how to write queries.

If some of the above resonates with you, then you may want to give Heap a try. Here at AdRoll, our goal has always been to make sound decisions based on customer data. We want to understand how people are flowing through our product, the features they interact with, the road blocks they encounter, and how this all influences their likelihood of success using AdRoll. While Heap is not our only solution, it’s a great start that will help us improve our data-driven processes. Here, I’ll share my experience using Heap and discuss the pros and cons.

Using Heap to determine average completion times.

Strengths

One of the key strengths of Heap is its simplicity and ease of use. It’s an extremely straightforward tool to pick up, even if you have limited experience with analytics tools. It comes with a good, basic set of analytical features that allow you to quickly understand usage patterns, overall funnel, popular (or rarely used) features, and even errors. You can also track events through time and analyze trends, look at success rates, and determine the average time on tasks. If you’re accustomed to crunching your own numbers and creating your own reports, you can also easily export data. All of this can be done with just a few clicks once you’ve configured all the events you want to be tracked. In addition, Heap retroactively collects data, so you won’t miss out even if you set up an event to track later.

Within the first month of our launch, we were able to identify several low hanging fruit to fix. For example, we determined that a sizable portion of form errors were from people leaving campaign or ad group name fields blank. As a result, we starting providing default names to help reduce these errors. We also determined that overall completion rate of the Ads step was the lowest (and also took the longest) of all other steps, at 53%. Armed with this data, we started designing new features to facilitate ad creation. For example, we’re currently adding a simple ad creator that creates a myriad of ads with just a few inputs, and we’re also exploring a feature to automatically create ads using images from the customer’s website.

Using Heap to determine completion rates for the Ads step.

Challenges

Although Heap is really easy to use, a lot of it is predicated on having a thorough setup. As I was performing analysis of our campaign setup flow, I realized that there were many things where either the setup was incomplete, or never touched. We overlooked some links, especially those in modals or drop downs. As a result, it became more challenging to quickly determine funnel in our multi-step product. While some of these were easy fixes, some solutions required tracking APIs and were a little more involving to set up. For me, the lesson learned was to be diligent when setting up what to track. It’s easier to be comprehensive and plan ahead than to add events retroactively.

Another aspect to keep in mind when using Heap is that this a great tool for tracking click events. However, clicking on a button or link doesn’t always translate into a completion of a task, especially with more complicated flows. Clicks indicate some level of intent. But, whether the click was deliberate or accidental is not always easy to say. For example, we have a feature in our campaign setup that allows users to create ad groups, or sub-groups within a campaign to further customize audience targeting. While we can easily see the number of people that clicked on the feature, with Heap alone we were unable to determine how many actually launched a campaign with multiple ad groups. In these cases, it’s best to recognize the limitations and pair Heap with other analytical tools, or to work with your analytics team to get the most accurate picture.

Having a comprehensive event tracker set up is very helpful, and luckily Heap’s UI is fairly straightforward.

Quick Tips

As with any tool, there are advantages and limitations. Understanding those fully will help you glean the most out of your analysis. Here, I’ve listed some quick tips:

• Be diligent about what you track. However small the feature may be, comprehensively setting up events upfront will reduce the chances of you running into road blocks later on.
• Forget to track an event? Don’t fret — remember you can add this event later and it’ll still be able to give you data.
• Define what success is for you, which can be for the overall product and also for smaller sub-tasks. For example, although ultimately we were looking at overall campaign creation rate, we also wanted to see how people were succeeding or failing in each step and each input.
• Try to track API calls as much as you can in addition to events, this will provide more flexibility and detail in how you can monitor usage.
• If you have a complicated product, plan out how you want to run each report first. I made an outline of the reports I wanted along with the required criteria, which helped me review the process for accuracy and also sped up my tasks later. This helps because sometimes there could be multiple ways of getting at the same data, and you want to be sure you are consistent every time you regenerate the report.
• Save your audience segments. For example, we saved our segments for external users, all users, and internal users and for easy filtering.
• Create folders of saved reports. It’ll save time by allowing you to quickly regenerate reports and compare trends without having to reconfigure your report again.
• The “Funnel” section is great, but don’t forget about the other tools. I found the “Graph” section particularly interesting as well because it can show you trends over time, such as success rate, time spent, count of users, etc. Paired with the funnel data, it can yield some interesting data.

Conclusions

Overall, Heap is a great tool to help you monitor your product usage. Its easy to use and offers a sufficient toolset to visualize and analyze your data. As with other tools though, planning ahead and understanding its limitations is important and will help you on your way to a more data driven environment. Also, experiment! Part of understanding a tool is trying things to see how well they adapt to your needs. You may find that the limitations I mentioned here aren’t so much of an issue for you, or, you may find additional integrations to pair Heap up with for your ultimate analytical needs.