A Real Time and Data Virtualization Case Study
By Elisabeth Strenger
Cloud Data Insights (CDI) met with Ryan Fattini, who runs data engineering and data science at CITY Furniture, and Ravi Shankar, SVP and CMO for Denodo, at the Gartner Data and Analytics Summit in August.
Ryan told the story of how CITY Furniture took the success of a real-time data system for sales and extended it across multiple departments.
The journey begins with a software engineer and an IBM mainframe and ends with a data democratization initiative. There are many interesting stops along the way–a streaming layer, an IBM cloud data warehouse, a miscellany of data stores, a data fabric, and data virtualization.
CDI: Four years ago, you were a software development engineer, and now you are an expert data professional with considerable influence. What was that transition like?
Ryan Fattini: It started back at the previous company where I worked as the full stack engineer. I built out their e-commerce platform and the application layer behind it. What introduced me to data or at least solving problems with data was that we were embedded in a marketing report. There were questions that needed to be answered.
One of our major vendors wanted to know activation patterns around the sales of smartphones. Vendors wanted to know activation rates, what fueled them, and basically what was behind the trends that were coming up. Nobody really had an answer to these questions so I looked into how you would solve this kind of problem.
It turns out the answer was data science. <laugh>. We built logistic regression models by taking demographic data against our activation rates in our cities– basically modeling. That was the start of my transition from being an engineer to someone who solves problems with data.
After building more models, I realized that the problem with data science isn’t building the models, it’s the engineering components. Six or seven years ago I was hearing about a lot of failures in the industry; data science wasn’t working.
Companies were hiring academics who could build models but had no idea how to move them into production. It’s an engineering gap. I realized that most of what we were doing was engineering, not just model building. You can’t do one without the other.
Now we have the hybrid machine learning engineer kind of a hybrid, which is what happened in the software developer area where the roles of back-end and front-end developers merged.
When I joined CITY Furniture as a software developer–there was no data team, no data warehouse, and no analysts, so I brought the same data-solving solutions to CITY. I found a couple of other engineers who were also interested in this kind of thing, and we started picking at problems using a data science approach with our engineering teams.
We were going rogue at first but were able to show the company that this was the future and that we’d eventually need to do predictive and prescriptive analysis. When we presented our damage classification model to the CEO, he said, “This is great…but what I really need to know is predicting retail foot traffic.” So we pivoted to forecasting retail traffic by day by store.
See also: 22 Top Cloud Database Vendors
CDI: As happened to many other businesses, COVID-19 made forecasting almost impossible. What happened to your retail traffic forecasting model?
Ryan Fattini: The model ended up being critical. For brick-and-mortar retailers, keeping stores staffed to accommodate traffic was extremely difficult. There was no more historical context to forecast on, but there were some underlying things that didn’t change, and that was weekday seasonality.
Saturday was always still the busiest, then Wednesday. We plugged the traffic forecasting model into the scheduling system, which helped stabilize the forecasting as we moved through phases of operating by appointment only, then 25% open times, then 50% until we were fully open.
Data science had proven its value to the company, and we now have a dedicated team of data science engineers–another hybrid role.
CDI: There is a continuum of workflow and skill sets that is the typical breakdown between data science and DataOps or data engineering. That hybrid role could be the key to bridging that disconnect.
Ryan Fattini: We do have two academic data scientists researching potential models, but we also need engineers that can build operational models that are more connected to the business and can be delivered in three months.
CDI: You’ve set the business and cultural context for us. What can you share about the technology challenges City Furniture faced in becoming more data-driven?
Ryan Fattini: The starting point was an IBM mainframe that pulled data from almost a hundred systems. It had been set up in the seventies, so it had data structures built under…
Read the rest on CloudDataInsights.com
