From 0 to 60 (Models) in Two Years: Building Out an Impactful Data Science Function
In 2018, I was hired to help WW build out a data science function. At the time, there was just a single junior data scientist and zero models in production. Today, we have a thriving, talented team of nine data scientists, more than 60 live models,* and various data products in the hands of millions of WW members. When so many data science teams fail to reach their potential, what are some of the factors that led to our success?
*Actually, 83 models, including a set of 26 core models, some of which are separately trained and deployed across 14 markets. “From 0 to 60” sounded cooler.
Impact manifesto and executive support
To spin up a new team, you need executive support. Without it, you will fail. However, many organizations have hired data science teams with support but without a clear purpose — more to keep up with the Joneses (“everyone else is doing it”) than anything — and, ultimately, those teams fail (e.g., here and here). At WW, it was different. We had deep support across our executive committee (EC, our C-suite) and alignment and purpose of why we needed the team. Let me explain.
In February 2018, Weight Watchers, as it was then called, decided to do a reset. The company wanted to have a much larger impact on the world, to broaden its scope to health and wellness more generally, and so it set out a crisp, new mission: to “inspire healthy habits for real life. For people, families, communities, the world — for everyone.” To accompany this vision, the EC drew up an impact manifesto, an important document in our culture that not only sets out what we are trying to achieve but also how.
One of those hows was to “personalize all that we do.” The EC recognized that personalizing a member’s experience (such as through the food, fitness, or mindset elements of the WW program) would increase the member’s success and, further, that a data science team could help deliver this. Having that top-level support and alignment, without doubt, set us up for success.
A pragmatic approach
Data science teams often fail because they tackle the wrong problems and are not focused on impactful outcomes. One trait we hire for is curiosity. However, smart, curious staff can easily fall down some rabbit hole, trying to scratch an intellectual itch, and get sucked into some hard but inconsequential side problem when a simple alternative might exist. Our team has always been very pragmatic. We focus on the problems that directly impact the member experience or the top or bottom line of the business. We don’t do R and D. We’ll favor tried-and-tested off-the-shelf approaches where possible. No need for deep learning if a logistic regression can match the performance. This is not to say that we don’t use some of the latest innovations, but in general, we leverage simplicity so that we can focus on the hard (sub)problems that really matter, the true cases where you do need to do more research and exploration to make an approach work.
Self-sufficiency and complementary skills
As we grew the team, we hired for a team-level skillset. That is, while we hired smart people (of course) — seniors with broad experience or more junior members that exhibited huge growth potential — we hired those people for skills that were missing in the team as a whole. As such, our team is now very self-sufficient. We have talent that spans sourcing, transforming, and working with data; modeling and feature selection; data visualization; and business savvy, as well as machine-learning engineering skills. We stand up our models soup to nuts: from initial querying the data to final prod-level CI/CD (continuous integration, continuous delivery/deployment), caching, and monitoring. We are not dependent on another team, with its other priorities and roadmaps, to make our models available.
Self-service technology support
While we are self-sufficient, we are able to be so because of WW’s platform team. That team, which is truly world-class, provides us a suite of self-service tools that enables us to spin up a Redis cache, or to set up secrets securely, or to deploy our models via Docker containers to Kubernetes, knowing that they are secure and monitored, will scale, and will just work. Those tools empower us, and all the other teams at WW Tech, to essentially run as fast as we want.
Another aspect that aided our success is that there was an existing data lake, data engineering teams, and an analytics team. That meant, unlike some other data science teams starting up, we didn’t have to load (Extract, Transform, Load) our own data before we could build models and we didn’t get drawn into creating dashboards and ad hoc analyses. We could focus on building data products.
Primrose
As the team grew, we started to have multiple data scientists working on different models but independently solving some of the same problems. This was inefficient. We wanted to encourage code reuse and standards as much as possible. As such, we made a decision to switch to a monorepo in Git and to develop a common framework for model development, a system we call Primrose. Primrose is the subject of another of our WW Tech Blog posts.
Building out a framework for how we developed and configured our models meant that we essentially built a set of plug-and-play components that the whole team could reuse and contribute to. For instance, Primrose contains a set of connectors for how we read data from sources, a set of model families, a set of tools for feature transformations, a set of tools for writing predictions and other outputs to sinks, and so on — all stitched together by a configuration file, not code. This approach meant that team members could write less boilerplate code and focus on the more valuable aspects such as understanding the data and feature selection, and it meant that they could more easily peer-review (and learn from) each other’s code.
Quick wins, independence, and trust
In 2018, as a tiny brand-new team and business function, we had to prove ourselves. Teams across the organization were keen to work with us, but most had no experience working with data science; and, some had heard the horror stories of data science teams that plug away at a model for a year and ultimately fail to deliver. While we knew that, in the end, we wanted to get models into our members’ hands through our app — the position that we are in today — we also knew then that this was not the best starting point. To deploy our models to an app with millions of users requires designers, user research, integration testing, QA, and so on, and sufficient priority to get onto a product roadmap. That takes a lot of time and trust. The latter had to be earned.
Instead, our first projects tackled problems that we could completely control — back-office systems that we knew lacked inhibiting dependencies — and that we could deliver on. We built churn and return models that predicted which of our members might unsubscribe or resubscribe, which marketing could then leverage. We rebuilt a clunky legacy system that identified each sign-up as a brand-new member or associated them with a previous membership (in other words, new versus returning member), and saved a million dollars per year in doing so. Those early projects gave us the credibility to work with the broader teams.
Strong partnership with the product team
As we developed that portfolio of successful projects and built up our reputation, we started to work more with the product teams that owned the member experience in our app. For the reasons set out above, to be successful we needed a strong partnership with those teams.
We had to have an agreed-upon high-level strategy and then, tactically, align our roadmaps, coordinate and sequence handoffs, and develop the necessary infrastructure so that the app could leverage our predictions. That can be tricky. Fortunately, at WW, the relationship between the product and tech organizations is especially tight. Our tech head (CTO) and product head (CPO) report to the same boss, the chief digital officer; we have regular joint product and tech leadership meetings to assess progress, priority shifts, and discuss strategy; and we have a strong agile project management head who herds the cats, drives the necessary communication, and handles the resourcing and crew assignments to move everything along. Those all help data science’s work get on the docket, get built with partnership, and get out to our members.
I feel very proud of what the data science team has accomplished. There is an element of chance in all these things, for sure, but WW made its own luck by having the executive support, the data and data infrastructure, and a raft of teams thirsting for data science all ready for a new team as we joined the organization. It doesn’t stop here, of course. WW members, watch out! There are many more exciting data products coming up.
— Carl Anderson, VP of Data at WW
Interested in joining the WW team? Check out the Careers page to view technology job listings as well as open positions on other teams.
