Quality and dev-ops at the bank

David Hägg
Sep 18, 2019 · 4 min read

In 2016 I became a member of the team engineering the Collector Checkout product here at Collector Bank in Gothenburg, Sweden. I’ve spent most of my time here as a team lead, which means that I’m a backend engineer but it’s also my job to make sure we are a healthy team and that we feel accountable for the stuff we build.

This story is about the things we do to ensure we create value to our customers and at the same time taking full responsibility for the things we build. You build it — you own it!

Our board

We do a Kanban-ish process. Bottom up so the things at the bottom are the most important ones to close first.

  • We never pile bugs in a backlog. They end up directly on the board and are either fixed first thing or closed if they are not worth fixing.
  • We have deliberately used the word build over dev and QA over test because quality is a team responsibility and testers can help when we build stuff.
  • We are constantly experimenting with the board, this is why we have a white board and not only a digital one.
  • We try to visualize as much as possible to create a shared understanding of our work.
  • We have a specific lane for awaiting the outcome we are seeking, and a process to learn from our mistakes.
Image for post
Our kanban board — always work in progress

Planning

When we plan our work, also called refinements, we try to embed traceability from the start. We ask ourselves questions like:

  • What is the customer/user value of this?
  • How can we measure the value?
  • How do we know we have succeeded?
Image for post
Planning

Code reviews and pair programming

We encourage pair programming and always perform code reviews when we don’t.

This is a must have to create a quality culture and pair programming often makes our solutions better from the beginning since more brains on the problem tend to create better solutions.

Demos for the team

We often conduct small demos for the whole team in order to spread knowledge, but also to get early and valuable feedback. This is also keeping everyone involved so they feel that they actually own the product they are creating together.

Image for post

Enablers

Automated Infrastructure

We run the entire Bank in Microsoft Azure. Now that was a bit of an achivement being one of the first banks ever doing that with all regulatory requirements that we had to be compliant with. But we did it!

  • We own our infrastructure enabling us to scale up/down and build new things without interaction with other teams.
  • We run code to create that infrastructure for us.
  • Everyone in the team can do this.

Continuous Deployments

To be able to move quickly we are using Octopus to deploy our product into production. This happens several times per day and is no big deal.

  • Everyone in the team can deploy.
  • We always have releaseable code, and we use feature flags to be able to ship to production even if we are not fully through QA.
  • A simple way to roll back helps us if we do bad things.
Image for post
Octopus Deploy
Image for post
Launch Darkly for feature flags

Traceability

For a system that handles monetary transactions it is extremely important to keep a good audit trail of everything that has happened. We are using EventStore as one of our databases, but we also log extensively.

  • We have one place where all logs go and a tool to easily search them.
  • We constantly tweak log levels to keep our attention on the important problems.
  • We log a lot! And you should log more than you think, and when you have done that — put some more logs in.
Image for post
We use Elastic Search and Kibana/Grafana for our logs

Monitoring and alarms

Data and monitoring is a first class citizen in our team. It means that we measure almost everything we build in order to see if we achieve the outcome we are seeking.

  • The dashboards are central and is the first thing each team member goes through when arriving in the morning.
  • To keep an eye on the system while we are not at work we use PagerDuty and Slack.
  • Services such as Runscope and Cloudmonix ensures we are live from the outside world as well.
Image for post

I hope you had some useful thoughts out of this.
And as always — it’s is a never ending process of constantly being a little bit better

Image for post

collectorbank-payments

A blog from the people building the products at Collector…

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store