Photo by Oliur on Unsplash

How Monzo (the UK’s favorite mobile-only bank) is rolling out dbt

Go to any tech Meetup in the UK, and ask the audience to raise their hand if they have a hot coral Monzo card in their wallet — chances are most hands will enthusiastically shoot up. The love for this bank is palpable (and I mean, who loves a bank?). So I was thrilled when Stephen Whitworth, Data Engineering Lead at Monzo, signed up as a speaker for the second London dbt meetup.

Stephen’s team is currently in the process of rolling dbt across their data team — this includes data engineers, data scientists, and data analysts.

While a lone data analyst can certainly manage a dbt rollout (I did 👋), it’s hard to simultaneously manage the political landscape (try telling your finance team you’ll be spending your next sprint building infrastructure rather than reports) and the technical learning curve. Monzo’s approach to getting started with dbt is a much more common pattern in a larger organization: the engineering team manages the initial migration, rollout, and training, with the intention of handing dbt ownership off to the analytics team in the near future.

With that in mind, there are a few things that anyone owning a dbt rollout can learn from Monzo (with a lot of my own commentary mixed in).

Let’s start with the “why?” behind any dbt rollout…

As the number of data users grows, so does the need for a systematic approach to data management

Monzo has 25 data analysts embedded on product teams. 100s of Looker users (70% of whom use Looker weekly!). And a culture of data that starts with the CEO.

There’s a lot going very right with data at Monzo, and Stephen recognized the need to bring a more systematic approach to data management that would allow them to scale the things that are already working. He sees migrating to dbt is a leveling up of their analytics capabilities that will help them consistently deliver accurate, tested data.

Specifically, dbt will enable:

1. Faster development: Analyst productivity is important for Stephen’s team so he is particularly excited about setting up an analyst’s development environment to help them work effectively. It’s pretty easy to get dbt installed on an analyst’s laptop, and Monzo’s dbt project will only use a sample of their production data in development, making it easy and quick to run and test models that analysts are developing.
2. Increased reliability
3. Centralized knowledge: A challenge Monzo is facing right now is the challenge of “what does this thing actually mean?” Monzo needs that business logic to be stored in a way that is accessible for both technical and non-technical-users.
4. Machine readable analytics code

Those first three are common reasons we hear for why teams adopt dbt, #4 is something we hear less about, let’s go a little deeper there….

There is a big advantage in having a machine readable analytics code base

dbt tags are probably not the most exciting bit of functionality in dbt, so I was very impressed with how Stephen is thinking about using them.

The particular use case he referenced was using tags to indicate sections in the analytics code base that need attention with {{ config(tags=[“needs refactoring”]) }}. His vision is that one day they will be able to collect metadata on their analytics code base and report on things like: “20% of my DAG is built on cruft”.

They’ll also be parsing the yaml files in their project to report on how models in their dbt project are being documented.

Very cool to hear how the Monzo team is thinking about using dbt’s annotation functionality.

dbt is a steep (but worthwhile!) learning curve

While engineering is leading the rollout of dbt, it is with the intention that analysts will ultimately own all of the data transformation code that lives in dbt. This is a fairly common adoption pattern — engineers implement, but analysts take ownership over the long-term. And Stephen’s concerns about this adoption are familiar:

1. There is a steep learning curve: dbt gives analysts an enormous amount of control over data transformation code, data testing, and data documentation but in exchange analysts need to learn software engineering tools and workflows (command line, version control, etc.)
2. Low uptakes of dbt’s most powerful features: Stephen specifically mentioned being concerned that analysts wouldn’t take advantage of dbt functionality like testing. Fair concern, testing is one of dbt’s least used pieces of functionality and consistently the one most valued by our power users.

As an analyst who adopted dbt, I have felt the pain of the dbt learning curve. It is a HUGE help to have an engineering team that is supportive and will take the time to answer the many questions that someone new to software engineering workflows will have. Some tactics I would recommend to any engineer helping an analyst get started with dbt:

• Encourage analysts to pair with engineers, especially when it comes to becoming more efficient; workflow improvements, like setting your computer up to be productive, or keyboard shortcuts, are best learned by seeing how someone else does things!
• Prompt your analysts to join dbt Slack. We have a #beginners channel which is a very friendly place for new users
• Have empathy for your analysts. Engineers are taught to be process-driven. Analysts are taught to be outcome-driven. Analysts are used to being rewarded for the speed with which a report was delivered, not the repeatability of the code used to create that report. Your analysts are 100% capable of learning new workflows, but be understanding of the fact that you’re asking them to adopt a very different picture of what success looks like.

Here’s a great article Stephen shared recently about managing migrations if you want to go deeper on this topic.

Thank you, Stephen, for the great talk! I hope we can get an update on the Monzo rollout soon. Thank you, Dylan Baker, for organizing another great London meetup. And thank you to everyone who showed up.

You can watch the full talk here.