A (hopefully simple) comparison of analytics packages
I’ve spent a lot of time researching, installing and using analytics tools. Below is a summary of what I have learned from personal experience and from reading other people’s comparisons. I hope it saves you some time.
There are two things you should do before you start looking at analytics packages.
- Determine the questions you want to answer. These are things like:
- Which acquisition channels are driving users to my site or downloads of my app?
- How often do my users come back to my site/app?
- Where do users drop off in the purchase flow?
While your questions will be unique to your situation, most sites/apps care about the same stuff (acquisition, behavior, retention, etc). I’d recommend focusing on the 3–5 most important questions.
2. Set up Segment and a data warehouse
You will like Segment because it makes integration with other services easier (no more pasting javascript snippets). You will love Segment because of the data warehouse. This allows you to store all historical data in one place. If you have a warehouse set up, you can then import this data into new tools as well as run queries on the data. The packages below are great, but sometimes you need to query raw data to get an answer.
Providers
There are too many tools to evaluate them all. I deliberately limited the list to those that have a large client base, transparent pricing, and a free self-service trial. Please let me know in the comments if I should add another company.
Below is a detailed comparison (google sheet) and a summary of each tool’s pros and cons.
Pros: Very sophisticated cohort analysis. High usage ceiling on free tier.
Cons: Somewhat lacking other core analytics features. Usually need to use them in conjunction with another package. Pricing isn’t transparent.
Pros: Free. Demographic/interest/behavior data.
Cons: Difficult/limited event support.
Pros: Retroactive tracking. Visual event creator.
Cons: No Android SDK. Pricing isn’t transparent.
Pros: Drill down to individual users in some reports. Email retention/reactivation.
Cons: Expensive.
Pros: Retroactive tracking. Push/in-app/SMS/email messaging.
Cons: Less intuitive. Expensive if you add people plan.
Evaluation Criteria
These packages share a lot in common. In order to make a decision without being overloaded, I am comparing based on the most relevant and distinguishing criteria. For example, I’ve excluded funnel analysis as all of the tools do a pretty good job. Also, I am not going to include details like Kissmetrics associating event data with a user (and not the event) as I don’t think that this is an important differentiator for most use cases.
Event Definition
Events are the primary unit of tracking for most of the packages. An event would be something like clicked on a button, purchased an item or viewed a video. Data can be associated with events like amount of purchase or location of button. The important factor here is how difficult it is to instrument your site. The better packages will have visual event editors that allow you to create events by viewing/clicking on your site. Packages without this feature will require some coding.
Retroactive Tracking
Some of the newer tools will automatically track all events as soon as the javascript is installed on the pages. This is very helpful if your site/app has a lot of surface area and you are unable to anticipate every event you may eventually want to analyze.
Cohort Analysis
This is animportant way to measure retention and basically groups users based on certain characteristics. The most common way of grouping users is by time (e.g. when a user first comes to your site), and seeing how often users within each time group return with your site/app. More sophisticated tools allow you to segment the cohorts and/or create cohorts based any behavior.
Mobile App Integration
If you have a mobile application, you will have to create separate events for each app. Most providers have SDKs that do the basic stuff and some will have retroactive tracking or visual event editing for your mobile application.
User Identification
Most packages will allow you to identify a user after they register/purchase as well as tie the users previous behavior to the newly identified user. The better packages will allow you to tie the user’s actions across different devices/browsers.
Extras
Some packages will do more than just analytics. A common add-on is a retention/reactivation messages. Messaging is an interesting feature because most of the targeting and personalization data is already native in the analytics package. If you can work within the constraints of the package’s messaging tool, you can get a lot of functionality from a single provider.
