Becoming a data-driven start-up — Part 1: Analytics MVP using Power BI
This is the first part of a three-post series about Elaway’s journey to becoming a data-driven business.
· Part 1: Analytics MVP using Power BI
· Part 2: Building a modern data platform in Synapse Analytics (not published)
· Part 3: Establishing a data-driven culture (not published)
Analytics is said to be the “secret sauce” of the start-up success. Maybe that’s why Elaway hired me as the 5th employee, to work on the data-driven culture and creating “a magic data platform”. A month before starting my new job, I found my boss at my doorstep, holding “The Lean Startup” with a big smile on his face. My Elaway adventure was about to begin.
Elaway has been using the lean methodology from the beginning, which in short means evolving the business model and products by testing hypothesis through the build-measure-learn cycle.
Instead of spending months building something that might never be used, we start small and build the minimum viable product (MVP) based on some assumptions. Then we measure if that product is working as expected based on feedback, learn from the feedback and then repeat the cycle.
So when my boss asked me a few months later how fast I could get some automated analytics up and running, I decided to test the MVP approach, using Power BI.
What to build
At this point I had gotten to know the business and I had some hypothesis about what KPIs would be valuable to our team in a dashboard updated frequently. I was picking the “low-hanging-fruits” first, to minimize the first cycle time. My goal was to get these up as fast as possible, without compromising the data quality.
The power of Power BI
The choice of tool was easy, given the existing tech stack.
As you might have guessed, I’ve used Power BI before, which was also a big plus. But until now I had always connected to ready-to-use data models coming from SSAS or SQL storage, which was not my plan this time.
One of the flexibilities of Power BI, is that you can connect to a large variety of sources directly, do the data transformation in Power Query/M Script and then have the ETL and data model stored inside the Power BI Service, without anything on the outside. It will then look and feel more or less like an SSAS cube.
If one of the data sources went down, the whole model would stop refreshing. Also, if the data volume became too big or the transformations too complex, this solution would not work. Still, my hypothesis was that this would be good enough for a while, giving me time to get a proper data platform in place.
The building part
I explored the sources and data quality through the Power Query Data Profiler and figured out the needed business logic while building the model. My colleagues were helpful as always, in explaining the functional stuff when I was lost.
It was a lot of fun getting to figure out the technical solution on my own by trial and error — with the good help of my BFF Google of course. Never before had I felt the same level of ownership to a task.
Getting the Pipedrive/CRM data through Json and M script was the hardest part (due to an API result limit), but other sources came in with just a few clicks, like Google Analytics data, through the custom connector.
Power Query made it possible getting something working fast, but the M script generated behind the scenes are not easy to read even with well-structured code. And when building a model this way, you can’t avoid having to write some M script. I had to remind myself often that this was just a temporary solution and that no one should ever have to read the code except for me.
As I had feared, the model was becoming very slow. I had some complex transformation, as well as many dependencies between tables, pulling a lot of laptop resources, since Power BI is a desktop application. Doing one small update, could trigger a model update on many objects, and with my 16GB of RAM laptop, it could be a long coffee break before I could continue my work. At least the bad performance was only on the developer side, and never on the user side.
I did some adjustments to the model before launcing it, to make it bearable to work with until I had a middle storage in place. I identified the largest bottlenecks through the Diagnostics Analyzer, and had to remove one of the largest and most granular data sources- the charging sessions.
Launching
After a month or two, the first model was ready, with just a few reports on top. The reports were distributed through MS Teams, with a separate tab for the given team.
In the weeks following, I got plenty of feedback and feature requests for the next versions. But hey, why just listen to what people say when you can see what people actually do? I set up a User Metrics Report, AKA “stalking report,” showing who used what in all reports and tabs available.
Both user feedback and stalking users have been beneficial when deciding which report parts to keep/remove and which ones were used more frequently and should have some more work put into it.
Takeaways
This approach worked very well for us. The MVP way of developing helped me to see the absolute minimum I could deliver on, pushing me far away from the “perfect”.
I think viewing those first reports helped the team to see some of the possibilities that lies within data and analytics, which is an important step in becoming a data-driven business.
It was easier to set future targets once we knew the baselines. Some of the available data became key results in our company.
The first version also raised our data quality awareness, by painting the shit-in-shit-out picture. I’m guessing this is an ugly picture in most lean start-ups at some point.
The road ahead
What is good enough today, may not be good enough tomorrow. After launching the first version we’ve gone from 7 to 50 people, and the expectations and needs for data and analytics are evolving into something much bigger than what this direct-source model can provide.
We still have a long way to go until we have become a fully data-driven company. Luckily for me, I’ve gotten my awesome colleague Cathrine Wilhelmsen onboard, and together we now make out the Data & Insights team. In the second part of this series, we’ll tell you about our giant leap towards a modern data platform, using Synapse Analytics. This will open the door to data, analytics and machine learning as a service to our product teams. Stay tuned!
