Applying “Data Science Triage” to treat context-switching pains
At Root Insurance, our data scientists align with cross-functional product teams, but we are empowered to make long-term investments to develop fundamentally new ideas. Data science is guided by exploration, review, iteration and, on occasion, edifying failure — an approach that leads to the best outcomes. This emboldens our team every day to be thorough, thoughtful, and innovative.
This philosophy can result in timelines that are not always compatible with project roadmaps and business needs. But, it does not have to be this way…
Context switching can be perilous
In computing parlance, context switching involves diversion from the current system state, so the CPU can favor higher priority tasks. To pause a task, the CPU caches system instructions and data pointers. To resume it, the CPU refers back to the cached information.
Many engineers, data scientists, and analysts also recognize context switching as a euphemism for pivoting from one task to another, usually at the request of product owners and stakeholders.
Of course, humans are generally less efficient than computers. We lose days in caching the system state — documenting, preserving, annotating, etc. Likewise, we lose time resuming it — switching gears, reestablishing computing environments, relearning concepts, and more.
When the cost of transitioning becomes too great, efficiency suffers.
To complicate the issue, high-priority tasks are often assigned to our best people, who can find themselves frustrated and underutilized in a cycle of context switching. So, is an all-or-none policy best when it comes to context switching?
Always switch or always finish?
To motivate this discussion, we developed a simulation assessing two possible decision policies.
Always Switch: A contributor always switches to the highest priority task.
Always Finish: A contributor always finishes their active task, then switches to the highest priority task.
The design and implementation of this simulation can be found on Root’s brand-new public GitHub repository.
While the results illustrated above are sensitive to assumptions, there are several results that hold under various studies.
- Always switch becomes less effective as the cost of context switching increases.
- Always switch is less effective than always finish when the cost is assumed to be greater than a few days.
- Even in scenarios where always finish proves more effective than always switch, the average value of the task that is actively being worked on is generally lower. Likewise, the average value of the highest priority task that is not actively being worked on is higher. This might explain the temptation of the always switch policy that is inadvertently adopted within many organizations.
Why always doesn’t always work
This simulation assesses two possible decision policies: always switch to the highest priority task versus always finish the present task before switching. Of course, things are not so straightforward in practice, and an always approach often isn’t realistic, either way.
Although limiting context switching usually adds the most efficiency — and thus business value — there are times when this strategy fails. This happens when project lifecycles exceed lead times for ad hoc, business-critical needs, such as when meeting a sharp deadline, settling a customer question, or responding to investors.
What’s more, although businesses measure performance in quarters or years, leaders and product owners are often incentivized to deliver on much shorter timelines. This leads to prioritizing short-term returns over long-term strategic work such as fundamental research.
Engineering a solution
In order to balance committed data science exploration with practical business needs, the Data Science Team at Root looked to our engineering counterparts for inspiration.
Root’s engineering organization assigns engineers to triage on a weekly rotation. Triage engineers protect teammates from would-be disruptions by being the first line of defense for any unexpected issues. They address failures, fix bugs, tune monitoring and alerting capabilities, and address backlogged tickets, so their teammates don’t have to be diverted.
This type of program is broadly accepted within industry. But, little has been said about its implications for data science.
Data Science Triage: an organizational salve for the pain of context switching
Much like engineering triage, a Data Science Triage approach provides efficiency and minimizes disruptions.
Data Science Triage addresses tactical tracks of work, including anomaly investigations, root cause and corrective actions, customer and marketing support, and more.
These core elements have been vital to the success of Root’s Data Science Triage:
Rotation: We assign a single data scientist to triage on a weekly rotation. We recommend allowing 2–3 months between an individual’s triage appointments.
Backlog: We maintain a communal backlog in a central location. A data science manager or senior data scientist is responsible for ensuring that it reflects and prioritizes the needs of all business units, but contributions from all data scientists are encouraged.
Atomicity: We strive for well-defined problem statements that are achievable within a week and for clarity around how results will be deployed (new products, tools, and/or insights).
Consultation: On the Friday afternoon preceding their triage appointment, our on-deck data scientist meets with their manager and subject matter experts to get context and define problem statements and acceptance criteria. Timing is crucial because it limits interrupting existing work, allows the data scientist on triage to hit the ground running on Monday, and affords the weekend for creative percolation.
Community: We maintain a Slack channel to communicate about active triage projects. Each week, we highlight the upcoming triage project and celebrate wins from previous weeks. For our team, this has become a bustling forum for exchanging ideas.
Mandate: We maintain buy-in from our data science leadership and stakeholders, who acknowledge that triage responsibilities might affect the delivery of other work.
Data Science Triage has other hidden benefits:
Knowledge sharing: Our individual contributors broaden their exposure to subject matter, technologies, and data sets and have the opportunity to interact with new stakeholders.
Onboarding: We’ve had success with onboarding new data science hires through triage. Doing so takes advantage of an already-curated set of diverse and atomic problem statements.
Proof of concepting: At Root, our Data Science Triage program has proven to be fertile ground for demonstrating the efficacy of more speculative ideas — away from the fulfillment of operationally critical tasks — and given birth to valuable new tracks of research.
Learn more about Root Insurance.
