Analytics and Data Driven Development in Apps
Analytics and Data Driven Development in Apps
Posted on February 17, 2016 by Imre Kelényi
Skyscanner is a data driven company at its core, so our decisions are driven by data. This is a fundamental thing for all Skyscanner engineering teams, including mobile app developers. Data is required to validate experiments, it feeds our alerting and monitoring systems and is the source of various metrics, including KPIs. In all cases having complete, timely and accurate data is essential.
What kind of analytics and logging tools do we use to collect this data in our native apps? And how do we collect these in a scalable way? Let’s get started.
If you release an app, you can pick from a myriad of analytics tools to help you get information about who uses your product, and how the use it. Our apps are no exception: we send events to both third party and in-house event logging services.
You might wonder why do we need so many different tools. The main reason is that each one of them has different benefits; and one tool might fit a team’s needs better than the other. Google Analytics is great for following larger trends and KPIs. AppsFlyer can collect all data (including app installations) that is needed for in-app ad targeting. However, most of our internal tools work best with our in-house event logging services. We also like to experiment with new tools, so the toolset can change from release to release. Anyhow, this post does not aim to give an overview of the available analytics tools (there are plenty of other articles about that ).
When it comes to integrating analytics into apps, the first problem we needed to tackle was logging to multiple services. Naturally, we didn’t want to litter the code with duplicated event logging calls just to send the same event to several services. We have created a centralized analytics mediator framework which acts as a proxy and handles all event logging requests. Requests containing the event details are converted to each analytics providers’ own format and dispatched to the given service. This way a single call can trigger multiple logging services and the framework can easily be extended with new tools. Event filtering is also needed since we don’t want to send all events to every service: an App Start event might be relevant for each provider, but we don’t want to log each button tap in all services.
Automatic Event Logging
Speaking about our development workflows and tools, speeding up feature development and removing friction caused by the tooling is one of the most important factors. In the case of event logging, this means that it should happen almost automatically, with minimal effort required from developers. To achieve that, we divide in-app events into two categories:
- UI interaction: events that are triggered by user interaction, such as tapping a button, dragging a slider or scrolling through a list.
- Business logic or data events: everything else, i.e. events that are not directly connected to the manipulation of UI elements. This category includes the app start, a completion of an API call or the business critical point when the user initiated booking a flight.
Business/data events are logged explicitly by manually adding logging calls into the codebase. The good news is that less than 20% of the events we send from apps fall into this category. The rest belongs to UI interactions for which we came up with an automatic solution.
The main issue with UI interactions is that there are a lot of things that can be logged. Basically all controls on the screen are potential event sources, often triggering several different types of events. You could argue that we just need to track a few key interactions, but we have realized that having a high event coverage can be a great asset:
- We can precisely replay complete user journeys. This is not only useful for analyzing how the app is being used but can also be invaluable for debugging
- Apps have a longer release cycle (especially on iOS) and in many cases we just can’t wait until the next release to add an event. Also, it’s sometimes not possible to determine in advance what needs to be tracked to validate a certain assumption or gather data for the research need for some new features.
So how do we achieve this? Let’s take an example. The following figure shows a screen from the Skyscanner Flights iOS app and the event
SearchResultPage ListItem Tap. This event is automatically triggered when the user selects an itinerary card on the flight search results screen.
To make this happen, the only task of the developer is to assign an analytics name to each control that should be logged. No other extra code is needed. Of course there is no magic here. We have extended many of the platforms built in UI classes to automatically perform the logging. On iOS, this mostly means categories (e.g. a
UIButton+Analytics) and some custom classes. On Android, we also use custom view classes and hijack event handlers so that logging is performed in addition to their normal behavior.
The other clever thing here is how the name and properties of the events are generated. We wanted to have a consistent and clear naming convention for our events that also follows the hierarchy of the UI. In the name of the event
SearchResultPage ListItem Tap, the Tap part is the actual UI action that has happened. The other two components are the name of the direct event source (
ListItem) and its parent’s name (
SearchResultPage). This name is generated by our analytics framework. When an event is triggered on a view, the framework walks up through the view’s ancestors and collects the analytics names in the hierarchy. On iOS, we use the responder chain as the event hierarchy. On Android, a similar chain is formed by connecting the activities, fragments and their view hierarchies.
Some events displayed in our in-app analytics debugging tool:
In addition to the name resolving, this approach also provides an event context (set of additional event properties) at any level of the chain. The properties of an event are the aggregate of all contexts added at any level. In the following figure, you can see that the tap event also includes the top level application context, which contains, for instance, the user preferences (e.g. the user’s language).
Challenges with Automatic Event Logging
Automatic logging and generating a lot of events surely sounds nice but is there a catch? Here are two possible issues and our answer to them:
- Does sending so many events hurt the client side performance or bandwidth? — At the time of writing, a typical five minute session of opening the app and performing three different searches while navigating between screens and changing various search parameters generates about 100 analytics events. The total traffic sent to and received from various analytics services is in the 300 kB range. Although this is not negligible (and we are constantly working on improving it), it is still a relatively small amount of data compared to the cost of a web browsing session. Also, almost all analytics tools support batched updates which means that only a few actual HTTP requests are used.
- Since event names are bound to the UI hierarchy, if the hierarchy changes, event names change as well. How do you deal with this? — It turned out that in most cases, if the UI hierarchy changes, it also changes the user experience, which should be analyzed in a different way. It is kind of a paradigm shift, but accepting that if something moves in the UI hierarchy, then its analytics events might change as well, works well. If an event tied to a UI element is business critical and should never be changed, we can still use a custom data/business event with a fixed name.
Before adopting the new analytics framework and the automatic UI logging concept we struggled a lot with inconsistent event names, missing event properties and unlogged interactions. The introduced system has really helped in avoiding these and saving precious development time. Anyhow, logging is really just the first step in the life of data. After it has arrived to Skyscanner’s data stores, it is transformed, visualized, analyzed and fed to many different systems. And with several tens of millions of events logged per day, this is a complex problem our data team is delighted to tackle in innovative ways.
Originally published at codevoyagers.com on February 17, 2016.