How Draft Tackled a Data Mess and Improved LTV 17% in Less Than 2 Months
Summary: Mobile app Draft leveraged Method Mill to make data-driven decisions to increase user LTV by 17%.
- Platform: Android, iOS
- Location: New York City
- Category: Fantasy Sports
Three Primary users:
- Nicolo- CTO
- Justin- Product Manager
- Dave- Analyst
With the NFL season about to begin, Draft’s biggest obstacle was increasing their user’s LTV before the season start. Football season may happen every year, but for a young company like Draft, there’s only one chance to get it right, and without the ability to quickly and reliably track key performance metrics, they were flying blind — at a critical time.
The team knew that the best way to accomplish their goal was to be data-driven with their marketing, product, and sales decisions.
Draft is a mobile only, daily fantasy sports app, where users can challenge their friends or strangers in head-to-head matches for cash. The team knew that their success was contingent on their ability to measure these KPIs and they had a to find reliable solution quickly, that would work with their existing data sources.
The data sources Draft had to work with:
- Localytics, an analytics tool that focuses on providing apps with the data needed for attribution and remarketing.
- PostgreSQL, the internal company application database powering the app. Application data is the data living on Draft’s server.
- AppBoy, a service for sending and managing push notifications The team at Draft relies heavily on this service as one user session can generate up to 6 to 7 notifications.
- mParticle, a data integration layer that captures all client-side data. Client-side data refers to any action a user performs in the app (including button clicks, swipes or even shaking the phone).
In order to increase LTV, Draft decided to follow a 4 step process:
Step 1: Measure/analyze what factors were impacting LTV
Step 2: Based on the analysis, make key changes to the app in the interest of improving user acquisition and retention (churn)
Step 3: Re-measure and analyze whether the changes were effective
Step 4: Repeat
Step 1: Measuring virality
Before making any big decisions, the team had to determine what elements of the app impacted LTV. They were able to determine their K-Factor with tools listed above, but couldn’t use these tools do anything to change the number for the better.
Draft had the following questions:
- How does LTV differ between MLB and NFL users?
- Is there a relationship between the amount of money a user deposits and how many friends they invite to join the app?
- What percent of challenges are being accepted each month?
The team’s analyst, Dave, couldn’t answer these questions because no of their data sources provided a holistic representation of the app. Their data was trapped in silos across each service. The picture of the app’s performance would become much more clear once their data sources were joined into a single queryable database.
Draft turned to Method Mill for a fast, no-integration solution. Their CTO Nicolo plugged in the mParticle, Localytics, AppBoy and PostgreSQL credentials to Method Mill, and their client side and transactional data were immediately joined in a new queryable database (Amazon Redshift). This was all done without the overhead and complexity of additional code or third party dependencies.
That same day they were writing SQL queries and finding the answers to their most critical business intelligence questions.
“Integrating Method Mill could not have been simpler. I just inputted our credentials and the data was in Redshift, and ready to query in a few hours. I loved that there weren’t any SDKs or APIs to worry about, and all analytics queries now happen away from our production server.” -Nicolo
Now for the answers to those burning questions:
- Pre-season NFL users were not challenging friends to new games, they were were just beginning random drafts with strangers. This is a key finding because NFL users were skewing the k-factor lower.
- With the challenge question, they learned that 75% of the challenges being sent out each day were being accepted, and that the number had grown 5% each month since launch.
- Users that had deposited more than $25 were 4X more likely to invite a friend within the first week of using the application.
“Being able to query both transactional and event data at once is a dream come true. There are finally no more unanswered questions.” -Dave
Another great benefit to the team was that Dave was able to save his queries, and the team could run them and generate reports any time they liked.
Step 2: Making informed changes
“After using Method Mill, almost all of our changes to the app are based in fact. There’s no more shooting from the hip.” -Justin
Equipped with knowledge about what affected LTV in their app, Draft made a multitude of changes, including:
- Encouraging their NFL users to challenge their friends through push notifications at key times during the week.
- Encouraging users early on to deposit money early, as well as emphasizing that users challenge friends for cash through the UI.
Draft also decided against certain proposed changes. Sending secondary push notifications encouraging users to accept challenges that have been waiting for a long time was discarded. Draft learned that reminding users to accept a challenge was frustrating and resulted in decreased engagement.
Step 3: Re-measure the data
Draft’s case is a perfect example of how small changes can have significant impact on a product’s success. Just a few alterations improved LTV by 17% in a remarkably short period of time, and now the team is optimistic about this key performance indicator continuing to rise in the future.
Step 4: Repeat
Draft has very ambitious plans to become even more data driven with Method Mill. Here are the steps they’re taking to improve their product:
- Integrating paid install data from Tune in order to track every step of a user’s journey from the initial campaign to their first cash deposit.
- Connecting thier data in Redshift to a data science tool like R or Python to extract even deeper insights by taking advantage of the latest machine learning tools.
- Experimenting with leveraging Method Mill’s ability to push to non-relational analytics databases like Elasticsearch, and then visualize that data with Kibana.
- Utilizing tools like Tableau or Chart.io as a visualization layer on top of Redshift.