Framing the future with next-level data and analytics
There can’t be many e-businesses today that don’t value, and act upon, insights gained from data analytics.
On the other hand, setting up the comprehensive, robust system needed to supply such actionable data is far from simple.
That’s particularly true for start-ups, where the priority in those early days is to create a viable working product.
Most such operations will have some reporting built in from the beginning. Often it will be basic; as the business grows and technology develops, the demands placed on the reporting and analytics suite start to strain its capabilities.
In this post I’ll talk about how we at MPB went about addressing just such a challenge.
When I joined, a little over two years ago, MPB was already a very data-literate organisation with a huge appetite for new insights.
It was also close to delivering a new software platform, which required a massive amount of configuration and testing to ensure our business data remained available and consistent.
So for the first time MPB had set up a dedicated Data and Analytics Team — three of us at first. While continuity was the immediate priority, this was also clearly an opportunity to review our existing systems and develop new functionality. And with that, the journey began.
Data, data everywhere …
As I got to know the business, I was delighted to see how heavily data-driven things were. This was a double-edged sword, however.
For instance, Tableau is a very capable data visualisation tool and we were using it to create dashboards for all manner of business functions. Unfortunately, we were also using it for a great deal of our data transformation. All of it, in fact. Our reporting process was becoming inefficient, hard to maintain, inconsistent, and neither robust nor scalable.
Tableau extracts were failing with growing frequency. Data complexity and the expanding requirements of a growing business were all adding to the load. This wasn’t going to be sustainable.
Moreover, there was no single source of truth. That was probably to be expected from any growing organisation which hadn’t previously had a centralised data function; but it meant there was no unified governance.
We were maintaining a plethora of unconnected data sources and this caused issues around efficiency and consistency.
Tableau reports had been built up over time by individuals across the business. Many of these reports had become unwieldy and difficult to navigate; documentation was limited. This hampered confidence and users were often uncertain about which source to use for a particular task.
Driving change
It was time for change but we couldn’t just say “build it and they will come”. The power of data is wasted if nobody uses it. We developed a five-point strategy to encourage confident data-driven decision-making throughout the organisation:
- Centralise and govern our data
- Simplify, consolidate and enhance reports
- Create full documentation
- Provide proper training business-wide
- Showcase the power of data.
Sounds great. Now what?
First, we hired the Data Engineers who would play the pivotal role, laying the groundwork for a scalable system which could adapt to future requirements.
The next goal was to create our single source of truth — a central platform where data could be governed, secure and consistent, while allowing efficient access and robust automation. This would future-proof existing capabilities, and allow for data to be manipulated, joined, stored and mined in new ways.
We chose Google Cloud Platform’s (GCP) BigQuery to load, store and transform our data. Another six challenging months of planning, testing, collaborative discussions and reappraisals, and we had a live data warehouse plus several key transformations, driving transactional, product and customer level reporting and analytics.
Building from scratch meant we could incorporate best practice from the very beginning, with no legacy hangover to consider. We harnessed the capabilities of GCP’s Data Catalog to build our data dictionary, governing our data within a central repository and ensuring robust outputs now and in the future.
Navigating change
Projects like the building of our data warehouse can be challenging enough to deliver on their own, but such things rarely happen in a vacuum.
For one thing, it was built in parallel with a major business-wide replatforming project. This meant our key stakeholders had limited availability, while our own team had to migrate reporting functions in addition to keeping the lights on for existing data capabilities and continue to support current projects.
It was important to bring stakeholders on the journey. Fortunately it was already widely understood that the current system was not scalable, so there was no issue securing buy-in. We kept stakeholders updated, delivered quick wins where possible — and made lots of promises about what was to come!
There is no doubt this was a challenging and fast-paced period but the team worked in collaboration to create a considered and future-proofed product.
Unleashing potential
We were proud of our achievements to date but in order to create business value, we needed to surface our data.
Our next focus was migrating dashboards, at the same time consolidating and streamlining their structure. For our analysts — who now had to migrate data and reports for the second time in 15 months — this called for a huge amount of perseverance and belief in what we were developing for the long term.
Dashboard creation now became much easier. Moving away from static data-heavy reports, visual dashboards became more dynamic and encompassed more data with less real-estate.
In parallel we had been adding new sources and capabilities to our data warehouse, allowing for holistic end-to-end analysis and enabling the joining-up of on-platform data with transactional and customer data. Storage of historical point-in-time data now enabled longitudinal analysis, KPI monitoring and statistical model development.
As well as reproducing function-specific reports, these new capabilities enabled us to launch five key “heart of the business” dashboards, surfacing macro-level performance and trends while enabling self-serve drilldown for immediate hypothesis-testing and insight.
Having launched these dashboards, we led training sessions, produced introductory videos and created top-level insight decks to support wider education, speed up adoption and increase user confidence.
Tableau was now not our only data end-point. Raw data was shared where relevant to avoid Tableau being a middle man to data access. Our Single Customer View was shared with our media platform to take our marketing to the next level.
Then, three months after the launch of our data warehouse, equipped with advanced capabilities, we took an ambitious leap. We embarked on and successfully deployed our first data science model, marking a pivotal achievement in our data-driven journey.
Through a multi-stage process, using machine learning and a series of optimisation algorithms, we supported the development of MPB’s dynamic pricing engine — the world’s leading proprietary pricing engine of real-time prices for more than ten thousand used cameras, lenses and accessories.
Mastering change
As with any new team, ways of working and best practices needed to be established. Daily 15-minute stand-ups (for which we have never in fact stood up!) and two days a week spent together in the office have helped to build trust and collaboration.
We had a collective and strong belief in what we were trying to achieve. We facilitated each other; unblocking and keeping everyone informed is always important, but is critical when moving at pace and delivering outputs at the same time as building foundations.
Data has always been key to MPB’s way of life and there had been a building wishlist long before my arrival. This, in addition to the increased excitement following the success of initial dashboard launches, meant there were a lot of requests.
To keep track of them, combine related items and allow holistic prioritisation across the business, we have launched a ticketing system. With 1,000 tickets submitted in 15 months there’s still plenty of work to do in prioritising and progressing requests, but we at least have an easily accessible way to keep abreast of every item’s progress.
We’re using Confluence as a one-stop shop for documentation and guidance. This is a wonderful resource, not only when onboarding new starters but also at those times when you need a refresh!
Success is a team sport
Our team has been scaling and now comprises four functions: Data Engineering, Business Intelligence, Analytics and Data Science. There are now ten of us and we’re planning to grow further.
Team ways of working and culture are key to the end-to-end success, particularly during a period of fast paced transformation; trust and collaboration form the foundation of our working environment.
We aim to cultivate a safe space where diverse perspectives and experiences are highly valued. Nobody holds all the answers and our strengths complement each other. This drives efficient and effective decision-making, while ensuring collective learning and development.
As a Data and Analytics team, we actively engage with our technical engineering peers, leveraging their expertise to enable and enhance our capabilities. Simultaneously, we partner closely with our business stakeholders, empowering and supporting them in their data-driven initiatives.
Our monthly Data Centre of Excellence meetings serve as a formalised collaborative platform, bringing together the data team, peers and stakeholders to review successes, define goals and set expectations for continuous improvement.
Looking ahead
Without a purpose, foundations are meaningless; similarly, data deliverables lack substance and reliability without robust foundations and well-established infrastructure.
We have an ambitious and exciting roadmap for the year ahead. We will continue to reduce manual tasks, increase data democratisation and data literacy, whilst being at the forefront of the changing data landscape.
Ingesting real-time data and leveraging data science and AI will enable us to enhance internal workflows and support MPB’s customer experience. Driving innovative data-driven strategies that significantly impact business outcomes and increase our focus on continuous improvement and refinement.
Continuous education of the team and the broader business remains pivotal as we aim to move swiftly. Our focus is on providing the expertise and resources to drive data-literacy and empower our business. The future holds great promise and we are excited about the opportunities that lie ahead.
For current open roles, please visit careers.mpb.com
Nikki Miles is Head of Data and Analytics at MPB, the largest global platform to buy, sell and trade used photo & video gear. https://www.mpb.com
