Why did we start investing in our own Data Platform development?
Developing our own data platform and culture within
What do people develop their own Data Platforms for? In our company, we had a few strong reasons to start investing in this direction. However, before we start, let’s spend some time discussing the business and solutions we had two years ago.
Dodo Brands 2020
Dodo Pizza was one of the fastest-growing and innovative Pizza Chains in QSR business. From the very beginning, we have been developing our own information system Dodo IS. As you can see, both business and IT are interdependent to each other, and Dodo company can’t be imagined without either of them.
We are really proud of Dodo IS and it is a great advantage of our franchisee offer. If you are a businessman in any country, you can buy franchises and you will get not only a Brand and a ready-made menu of products, but an information system will be also involved in all processes inside pizzerias. This system will show you the health condition of your business due to the unified IT heart of your business which accompanies all orders from mobile apps to the ERP module of Dodo IS Platform.
And at this step, we are getting closer to the data subject. Tons of information is generated by our stores in many countries. The first years of developing our Dodo IS Platform were focused on the tasks to help to improve business processes by automating operational parts, developing great products for getting orders from our customers via the web and mobile apps, and developing an ERP system with the manager’s admin panel.
I have to say that analytical things were more about developing the Reports Section, and the solution was great because every manager of the store chain could download reports about all characteristics of the business like Revenue, Stores Performance, Unit Cost, Labor Cost, etc. It had sounded great before we met infrastructure and logical problems.
H̵o̵l̵i̵d̵a̵y̵s̵ problems are coming
The first problem is about the technologies we were using at that time. The Reports Section uses a transactional database underhood. The main task of this kind of database is to work over operational processes and not analytics. As a result, we were getting more and more problems with the impossibility of building reports over a long period, for example, complicated reports created for more than 2 months. On the other hand, it’s a tiny task for analytical databases.
The second problem is connected to the structure of our business. As I said above, our company is developing by a franchise model. We have a vast chain of partners who opened their own companies and stores, hired their own staff and teams who manage stores, develop local marketing, do things around financial reports, etc. At the same time, there is a Headquarters that is focused on the support of our partners’ development and improving Dodo IS. In other words, the Headquarters is the provider of IT and Business solutions for the final legal entity that bought franchises.
Working with data is getting more and more in demand not only in IT but almost in every sphere. As a result, our company had to pay attention to developing analytics in order to remain a fast-growing and competitive business in QSR. We have started developing analytics within since creating the Business Intelligence team and looking for Analytical Tools. After some period of investigation, we have chosen Power BI as the main tool to improve our analytical approach.
At this step, I offer to skip a few years of developing analytics in our Headquarters and proceed to describe the main reasons to start developing a Data Platform within. Since we chose Power BI as an analytical tool, of course, we understood that this solution would limit us with the impossibility of scaling analytics up to the whole chain of franchisees. Almost all suitable BI tools at that time were licensed, so every user had to buy a separate license. Even though there were other solutions about licensing over user sessions, it still didn’t solve the situation with a great chain of our franchisees’ teams. Have you ever tried to calculate the necessary annual payments for thousands of employees? It’s a very funny exercise, I strongly recommend it. After that, imagine that you have to talk hundreds of business owners into paying extra money for the BI tool in case they pay Royalty Payment, and Dodo Information System is a part of the Dodo franchise.
Let me sum up all things around analytics in Dodo Brands at the time of starting active development of our own Data Platform:
- Headquarters has Power BI tool inside, but can’t afford to buy licenses for all users of Dodo IS;
- Franchisees’ teams can only use the Reports Section of Dodo IS which is developed by software engineers instead of analytics. As a result, all tasks of developing new features in reports are uncompetitive with creating new features in other product domains;
- There are different approaches for calculating the same metrics in different tools;
- Franchisees do not have access to advanced analytical dashboards that were developed by BI team in Headquarters.
The birth of changes in Data Culture
In parallel to developing the main Dodo IS, there were two more stories that affected all the next steps in developing data culture within Dodo Brands.
As I was an ML engineer who was trying to prove the necessity of developing data-based solutions in our company, I worked on forecast models to automate procurement in the supply chain. It is a very interesting and complicated goal, and the first months were spent changing processes, not developing an ML model. However, this article aims at other goals, you will be able to read about our automating procurement project in other articles in the future. Meanwhile, at a certain moment, I had to scale up my forecasting approach to the whole store chain. And I was thinking about how I could generate and provide predictions for more than 200 000 time series a day and automate this process to improve it repeatedly. So, I had to find some approach and a set of tools to support the whole lifecycle of data movement and prediction for the procurement automating project.
The second story was about creating a new data engineering team. We didn’t have any special long-term plans and strategies for developing a data culture in our company. However, we believed that we had to support several guys in different teams who were really interested in developing data engineering skills to keep them more focused than they could be in their product teams.
Each of us had our own goals, but after some time we decided to develop a common project. Our MVP should be very easy to describe: launching a forecast model over the data platform and scaling it up to the whole store chain.
Let me skip the technical details of those MVP projects, I just want to say that it was a real success. I saw that the data platform was an absolutely new approach in analytics for our company and that it should be used by every team in Dodo Brands. I was really excited by the solution and how fast I could solve such various analytical problems without wasting time preparing data from different sources and tools.
We decided to join forces and work on developing Data Culture within by providing tools, approaches, and knowledge that would help to develop data-based solutions in any product team of our Dodo Brands company. The guys from Data Engineering asked me to join as a leader of the Data Team (later it united 3 directions: Data Engineering, Business Intelligence, and Data Science), and we started developing Data as a Product, Data Platform, improving Data Culture, etc.
Moving in the darkness
I called the period of the next months “moving in the darkness”, and I had reasons for it. I remember that time when there were a lot of disputes between engineers about architecture, and many attempts to work out small projects with fast results and clear Returns on Investment. And of course, the most attractive idea in our heads was to create a data product that would be the most wanted in every product in our company.
We spent some time trying to connect the development of the data platform to any other product team plans to empower them. After a few discussions, we decided to take a few steps back and take a wider view. We started with the most important step in every technical project: the drawing scheme of our Data Product. It’s kind of a mix of data platform technical tools, data-based possible projects, and business units of our company.
This scheme helped us to avoid wasting time on developing too modern and complex systems in case we had fundamental problems with data. In our imagination, we thought that every colleague would have access to data and could analyze it. And that was a problem! In lots of books, you can find that the biggest challenge is to destroy data silos, but we had been creating them inside our company for years. The gap between the Reports Section in Dodo IS and Advanced Analytics created on Power BI that the Headquarters had was our real problem, and it was us who created the biggest data silo.
This illustration reveals that there are no ways to share data with our franchisee partners. The Headquarters has a team of business developers who help our partners to achieve their goals faster. They are like a locomotive of Dodo Brands’ business, and they didn’t have the tool to share the metrics and analytics with franchisees they had in Power BI. And we decided to solve several problems with one data platform:
- Replace the Reports Section with a united Data Platform for the Headquarters and Franchisees;
- Develop Analytics only by analysts without software engineers and their backlog;
- Destroy the gap between metrics calculations in different systems and the data silo;
- Build a data warehouse for creating datamarts and dashboards for everyone, and provide advanced data science tools to the teams that could develop cutting-edge projects over data.
Our new approach scheme was like that:
We delivered our vision and strategy to the heads of our company, got approval to invest in this, and started developing our Data Platform. I could say that developing such complicated projects is always connected with lots of investments, risks, and questions from other teams. You do not have to argue with anybody and only explain as simply as you can what advantages the company will get. However, it is important how people around you understand the projects you are doing, and it is worth having extra tools to reveal the progress of developing Data Culture within because it could take a long time. We have found one, and I can recommend it. It’s called Data Driven Maturity Assessment developed by Ruben Buitelaar. The author of this model has coped with creating a complicated system of estimation engine underhood and a simple interface for interviewing the customers.
An Introduction to the Data-Driven Maturity Model and Assessment by Ruben Buitelaar
Organizations often struggle to plan and measure their data-driven transformation. The Data-Driven Maturity Model and accompanying Maturity Assessment are designed to help organizations get back in control and make a successful transformation.
Maturity models can provide a blueprint for transformation. The DDMM consists of 5 stages of maturity: Reporting, Analyzing, Optimizing, Empowering, and Innovating. Generally, these will be the steps organizations take in their journey. From an organization struggling to produce outdated reports, to an innovating organization that leverages data to build a competitive advantage. Every stage is conveniently subdivided into 10 dimensions, so that you can see which areas are ahead and which are lagging behind. The 10 dimensions are Data, Metrics, Skills, Technology, Leadership, Culture, Strategy, Agility, Integration, and Empowerment. The DDMM is unique because it is very inclusive of both technical and non-technical aspects of a data-driven organization. This ensures you build up both technical capabilities and the right data-driven culture, both needed to succeed.
The Maturity Assessment is designed to measure the current state of the organization, across the 10 dimensions, with a conveniently designed questionnaire. It utilises ‘ranked scale’-type of questions, in which a user has to select the statement best describing the current state of the organization from a series of 5 statements corresponding to the 5 increasing stages of maturity. These statements are self-explanatory and easy to answer, you just make a comparison between your current state and the described state. Every dimension is scored individually, and the overall score is simply the average of all the dimensions. Multiple assessments can be averaged to create a more accurate and balanced assessment. The results of the assessment are compiled into a report with best practices for every dimension, which provides the right blueprint for creating a strategic plan with the right goals. You can periodically reassess to see your progress and realign your strategy.
The Data-Driven Maturity Model and Assessment are currently in use all over the world, and we’re developing convenient tools to aggregate data points and track progress over time. If you want to learn more, check out fiveten.io.
Dodo Steps history of estimation by this model
We have been using this Data-Driven Maturity Model and Assessment since we started developing our Data Platform. Of course, we are not at the top of this score and we still have a long way to go. However, at the current moment, we clearly understand our goals and how we are going to achieve them. We are thinking not only about destroying the data silo or developing one or two data-based projects. Right now we know how to make every product team in our company really data-driven step by step. We are developing strategies for implementing a data mesh approach and federated analytical structure in our company. About all these things and technical details our team is going to tell you more soon, however, now you can investigate our way of developing Data Culture within and increasing DDMM score over almost 2 years.
Since our company is trying to be as open as possible and I follow these principles, please feel free to ask me any questions or details about the article subject.
