Teachers’ Guide to Segment Migration

Recently the Teachers’ team, along with other teams at Prodigy, took on an initiative to transfer all its user event tracking to Segment within one quarter. Here lies the epic saga of how it unfolded…

Where did we come from, and where are we going?

Before Segment, we had our own events API.

A well intentioned, in-house solution to tracking events in an attempt to understand a user’s journey through the app and across different services. However, the API was unable to deliver on that goal as it couldn’t send data to all our analytical services and instead it led to a whole host of complications such as:

• sending inconsistent data to analytical services,
• difficulty in identifying a single user across analytical services,
• a lack of event data documentation, privacy control, and event observability.

What we need is to have all user actions tracked in one place so we can connect the user’s full journey, protect the user’s privacy by controlling the flow of data, and send consistent data to analytical services so we can make the best possible business decisions.

This is where Segment comes in.

Using Segment solves all the aforementioned issues. It allows us to:

• track consistently and reliably across our apps,
• track the user’s journey in one central location,
• control how we send that data to other analytical services.

This, in turn, allows us to gather trustworthy data to inform our business decisions to bring about the best, most engaging user experience, as well as protect our users’ personally-identifiable information.

With these concerns in mind, Prodigy made the decision to use Segment for user event tracking, and our in-house events API was to be deprecated at the end of the quarter. We would migrate all user event tracking to Segment before then, in order to avoid any data loss.

A Teacher’s Approach

It might help to preface with some context on what makes the Teacher’s team different when it came to tackling the Segment migration. The plan for all teams was originally to go through each app and replace all events API tracking with Segment event tracking. Simple enough? Except that while other teams had a few dozen events to swap out, the Teacher’s app had well over 600.

While already facing such a colossal problem, the team was dealt two more challenges: the project’s ownership was changed partway through and, since this was a large initiative that spanned across multiple teams, the data team also had their hands completely full and were unable to support us. This led to issues such as unclear expectations around the scope and timeline of the project. Continuing along with so much uncertainty would have led to less than ideal results, frustration amongst the team, and ultimately a deteriorated user experience due to lack of good data.

With these challenges in mind, the Teacher’s team decided to step back and redefine the project plan from the beginning.

Project Plan

When we inherited this project we were unsure of what it really entailed. We knew we had a bunch of tracking events strewn across the app that needed to become Segment tracking, but where in the app were these events, and which events were most important? These questions would prove to be instrumental in defining the scope and timeline of the Segment migration project.

Doing some initial calculations showed that with the amount of events the Teacher’s team owned and the size of the team, it wouldn’t be possible to migrate all event tracking from the events API before it was deprecated. With time ticking against us and less than four months left, how could we come up with a way to satisfy the project requirements while avoiding developer burnout and maintaining code quality?

We circled back to the actual project requirements and analyzed what we wanted to achieve at its core. We wanted to be able to consistently track a user’s journey throughout our app, which can be done by tracking the interactions a user makes with the app on the client-side. This distinction allowed us to deem the server-side events as out of scope and focus on tracking the frontend, user-generated events for the current initiative, bringing our number of events down to just under 600. Still massive, but more doable.

Documentation

The Spreadsheet

The next step was to find and document each of the events that would need equivalent Segment events. This is something every other team did in Confluence, so the Teacher’s team started our document there as well. However, it quickly got to the point where the Confluence page would simply not load up because of how large the document had become. So, we moved shop to Google Sheets. This turned out to be a blessing in disguise because it enabled us to do so much more in terms of planning and tracking our progress.

Finding 600 events and documenting their event properties is no simple task, but it was made much easier by developing a script that pulled the data out of the code and populated the Google Sheet for us (work smarter, not harder!). This alone helped ease much of the burden off of developers and saved a great deal of valuable time.

The Progress Tracker

Each row in the spreadsheet represented an event with documented details to create an equivalent Segment event (a trigger, event properties, and its location in the code). Each row also had a status, which mapped directly to a pie chart that allowed us to track our progress as we worked through this initiative.

Speaking as a developer, it was so satisfying to finish migrating a bunch of events and then updating the statuses in the spreadsheet and watching the “Completed” section of the chart grow. Implementing visual progress like this was great for developer morale as we worked through a project as cumbersome as this.

Time Estimation

Alongside this mapping spreadsheet and progress tracker, we also created a breakdown of the events that directly mapped to tickets for the developers to complete, and time estimations for each in developer days. This allowed us to efficiently plan out our sprints and work in a way that would allow us to complete the project on time while avoiding frustration and burnout on the team.

These events were then sorted into priority order depending on how important that event was to the user and to the business. Events that users triggered more frequently, such as those on pages accessible through the sidebar, as well as events related to referrals and memberships were prioritized so that even if we couldn’t finish the migration before the API was deprecated, we wouldn’t lose the data that is most important to us.

These documents also became a great point of reference to show project stakeholders. One such example is how it was used by the Data team. Since they lacked the resources to provide individual reviews, they could use this document to check on our plan and progress throughout the project, whenever they had time.

Commitment to Quality

Despite the tight deadline, the team had made the decision to commit to quality and consistency in our approach, so we wanted a thorough and well-documented process as we started getting into the code. Without this, implementing tracking for 600 events without any rhyme or reason would have quickly become a nightmare to work with.

To provide consistency we used a method we called the “anti-corruption layer”.

Within Segment, there is a tracking plan for each app that defines each type of event and the properties that are associated with it. This data will be consumed by data analysis services as well as our own data analysts, so it is important to ensure 1:1 compliance with the tracking plan.

The anti-corruption layer would allow us to do just that, while also making our lives easier as developers.

Each event type in the tracking plan corresponds to a track function in the anti-corruption layer that takes the event trigger and any required or optional event properties as variables. Not only did this mean we were sending each event type to Segment consistently for data analysis, but it also helped speed up development. For example, instead of writing out the whole function to send event tracking to Segment in every location there was a button click, we could now simply call trackButtonClicked.

And off we went, migrating each in-house API event tracker to a Segment event tracker. Before we knew it, it was the end of March, Q1 2022 and the progress tracker was all green.

The moral of the story

Through many trials and tribulations, the Teacher’s team emerged victorious. We used our newly redefined project plan, mapping spreadsheet, progress tracker, time estimation chart, and anti-corruption layer we developed to aid us in powering through this migration. Now that we have completed, we can develop features and implement event tracking in Segment for them as we go.

This event tracking in Segment will allow us to collect consistent and reliable user data for our data analysis services to consume. And having good, consistent, reliable user data will enable us to make better informed business decisions. It will allow us to create better features, better user experiences, and overall a better product that will keep users engaged and coming back for more.