The SaaS-y Power Couple That Will Save You 40% of Data Science Project costs
Why do teams of expert data scientists fail to deliver successful AI projects time after time?
The main reason for these failures is that many times, data scientists misunderstand the business needs. We put our biggest problems to be solved by people with advanced degrees in statistics, but without the relevant experience to comprehend the business use case.
Moreover, even if the business need is conveyed, a model building process can take months to years. If data scientists don’t check in regularly on changes in the business needs, the potential value of their work will rarely be realized.
The key to our success at WalkMe is a collaboration between expert data scientists and business analysts. We’ve found a unique way of implementing predictive modeling and scoring practices into our business applications along with creating a professional path for business analysts in the BI department. Most business analysts I worked with have high technical abilities and motivation to step into the data science field. This collaboration is a win-win for the organization. Our use of data science in this way, has significantly impacted our customer journey and enabled us to exceed our target KPI’s.
Data Collection and Cleaning
The perfect data scientist is an expert in three fields: statistics, coding, and business understanding. However, most data scientists out there aren’t perfect. The more technical they are, the less business understanding they have. So why are they the ones responsible for the data collection phase when building a model?
A business analyst, on the other hand, is highly knowledgeable regarding the quality of the various data sources in the company. Many talented and capable business analysts are interested in shifting to data science.
Through my experience with various data projects, I came to realize that when a business analyst is involved in data collection and cleaning — the earliest and most impactful stage on a model’s results — the fewer bugs in the model in the aftermath. So, pairs of data scientists and analysts together define the data and structure needed for building the best model to fit the business need. Then, the business analyst extracts the data.
Note that our business analysts are highly skilled in SQL, therefore they could take responsibility for the data extraction and cleaning process.
Model creation
This area is the data scientist’s expertise. They will need to sit down, by themselves, and research the best method of training the model with the highest accuracy. However, they aren’t knowledgeable enough to decide what attributes are a must-have and which are nice-to-have. During the model building stage, the data scientist and business analyst should meet regularly a couple of times a week, to review the progress of the model and raise questions about the findings so far.
QA
QA is, to me, the most important part of any data-based output. What is your model worth if its results are based on a mistake? There are two types of QA which need to be done: technical QA of the model — done by a fellow data scientist; and business QA. Business QA means not only making sure that the results make sense, but do they answer the business need we were tackling in the first place. The business analyst should review the model results, break them down to specific use cases and decide if they are correct.
An example of how QA saved a model, we once predicted quarterly sales ARR which looked slightly too high, only to realize that the feature used for the ARR was not yet converted to USD.
Maintenance
After we’ve come so far and presented the model to the various stakeholders, now the hard part begins. Revisiting the assumptions, which may and should change as time goes by, and tweaking the model itself if needed. This is the business analyst’s opportunity to learn the model and take ownership of it. A data scientist salary can run twice as high as an analyst’s — it’s in managers’ interest to use their time as effectively as possible.
How Much Did We Save With This Methodology?
After 3 successful projects executed with this model, we recorded the number of hours each person worked — data scientist and business analyst — and averaged it. Then, we compared these results with two benchmarks in the market.
We assume that a an expert data scientist’s cost per day (PHD with 7+ years experience) is nearly twice the cost of a business analyst.
*Assumption: Expert Data Scientist cost per hour = 2* Business analyst cost
In summary, this type of collaboration provides the benefits of a win-win situation: we’re effectively allocating the data scientists’ expensive time (and oh boy is it expensive if you have good ones), increasing accuracy of the resulting models, and providing career-growth opportunities for those analytical and bright individuals who want to dive deeper into data science.
