What should be the core value of a data pipeline team?
Reading notes of “Onward: How Starbucks Fought for its life without losing its soul.”
Recently, I finished reading the book “Onward: How Starbucks Fought for its life without losing its soul.” It records the history of Howard Schultz, Starbucks’s founder, turning Starbucks from losing business to finding its foot again during the 2008 regression.
Before 2008, Howard had been board chairman and left the daily executive duties for eight years (between 2000–2008). At the beginning of 2008’s recession, Starbucks’ core business indicators, such as per store revenue, were trending downward due to being too focused on financial results and aggressive expansion. During that process, Starbucks starts to lose its core value and focus.
Starbucks’ core value: Being the undisputed coffee authority.
Coffee is the business offering for Starbucks.
In 2008, many other Fast Food chains were also attacking the coffee business, such as McDonald’s dollar coffee and Dunkin Donuts’s offering. People even start to think those cheap coffee are just as good as Starbucks’ with the premium price tag.
Starbucks is in defense mode. They need to reclaim their authority position against other offers. Therefore, Starbucks took one of the boldest movements for any Fast Food business. It closed all of its U.S. Stores for one day and retrained all its baristas on how to brew the best espresso beverages.
It inspires me to think about the core value of a data pipeline team. I believe it should always be your data’s quality, reliability, and low latency.
Preparing data is precisely like brewing coffee. We would need good Arabica bees(raw data), collection, transportation, and roast them in a reliable and timely manner(ETL process). At last, if possible, we would also provide excellent coffee machines and other kitchenware (data applications) to help baristas to be efficient. Then it would be up to the baristas’ skillset to offer customers a tasty cup of coffee(business insight).
As a data pipeline team, we would control the ETL process and data application stages. For any data pipeline team to deliver business value, we should always put quality, reliability, and latency as our top priorities. That is our core value delivered to our customers: data scientists and business managers.
How could we build a low latency, high-quality and reliable data system?
It would be case by case for each pipeline team. But as long as we clear the goal and focus on this front. Eventually, with enough effort, resources, and talent invested in it, each team would achieve that to a certain degree.
But the important thing here is that a data pipeline team’s core value should be set correctly, instead of pursuing fancy technology and quick but lousy prototype to claim credits and political power (just like Starbucks chasing for the Wall Streets applauses).
Summary:
Using Starbucks’ example for their core value, we described why data pipeline teams should set low latency, high quality, and reliable data systems as their primary goal instead of quick but lousy prototypes.
