Author: Karim Meghji, CTO
The Challenge: Moving Fast without Breaking Things
As Remitly’s CTO, I have a responsibility to ensure our software engineering teams can deliver value to our customers and get their work done at optimal velocity. As a FinTech company, that’s not always easy. At Remitly, we treat our compliance, security and regulatory responsibilities with the utmost importance given the peace of mind they bring to our customers. With that said, these areas can introduce necessary friction to our engineering work. We’ve also continued to grow our team globally to keep pace with our business, and our technology and systems have continued to grow and mature as well. As a result, we’ve noticed an increase in organic friction due to more complex systems and a more complex organization. With both of these points in mind, we set off to better understand how we could counter these effects and enable an “and” outcome. How will we hold a high bar on compliance, security, and regulations, continue to grow our team, continue to grow and mature our systems AND reduce friction and optimize velocity across teams? This post is about the journey, which we are still on…and so expect some followup posts. :-)
First, Some Basics
Contemporary engineering metrics in use today were inspired by manufacturing metrics. You’ll hear talk about things like Lead Time, Cycle Time, Throughput, etc. which all have origins in measuring various stages of manufacturing processes. It makes sense if you think about it — we plan the work, we do the work, we check that the work is correct, and then we ship the work to customers. Measuring these steps (and their sub steps) can help us more clearly identify bottlenecks, roadblocks and non-optimized aspects of doing engineering work. Of course the bigger question is so what? Similar to how agile software development has embraced a number of “lean manufacturing” concepts, our hypothesis has been that if we optimize software engineering velocity, we can build more things that have the potential to add value to our product and for our customers. Key is the concept of “optimization” vs. “maximization”. Implicit in optimization is the inclusion of other factors, for example Quality. Just as in manufacturing you don’t want to produce a higher volume of poor quality products, neither do we when we are building software. So in some circumstances we may speed up engineering velocity but in other areas we may hold engineering velocity stable while improving another aspect (i.e. compliance, security, regulatory requirements, quality etc.).
So What Have We Done?
As a Data Driven company that thrives on learning and Continuous Improvement, we started by looking for easy ways to measure engineering velocity” with a supposition that by just starting to measure Engineering Velocity we would learn a lot, not only about where friction exists in our Engineering Pipeline (processes, systems, technical debt, dependencies across teams etc.) but also about how to approach improvement in the area of engineering velocity overall. And what we found was a growing industry of tools to measure various aspects of engineering velocity. We decided to engage with two companies that had different approaches to this domain and used different techniques to bring engineering metrics to life; our goals were to:
- Run pilots and learn about the metrics in a hands-on fashion
- Learn what measures were valuable or inadequate
- Understand industry benchmarks and how to use them in our day to day work
- Learn how to think about using metrics as “guardrails” for goal setting (or said another way, using them as a “healthy” state of a metric that we want to attain and then sustain)
We rolled out both tools in pilot form to a subset of engineering managers and leaders and held multiple feedback sessions where everybody had equal time to share their thoughts on what they liked and concerns they had about rolling out engineering metrics to all engineering managers and engineers at Remitly. We also solicited feedback on these two tools specifically. Finally, we followed the same process with our executive team (they weren’t using the tools so we summarized the metrics approach and our learnings) to hear their feedback given 1) the importance of optimizing engineering velocity for the growth of the business AND 2) the protection of our customers through our compliance, security and regulatory programs. That leads us to where we are on this journey!
What’s Next?
Based on all of the feedback we gathered, we decided to continue working with both tools as they seemed to complement each other well. We will continue our roll out more broadly to all engineering managers and engineers as part of continued learning and improving our understanding of how these metrics work and how we can optimize engineering velocity at a company and team level over the next quarter. We decided to initially focus on cycle time and have set up a guardrail target for this measure which Jordan (our VP of Engineering) and I will pay attention and study more closely. Furthermore, we will do the following:
- Establish baselines of the these metrics at a team level,
- Learn about opportunities to improve our engineering processes and rituals,
- Find hot spots where we should reduce our technical debt
- Learn more about the quantitative impact of having engineers focused and uninterrupted
- Learn more about the impact of cross-team dependencies and work
- Look more closely at other measures (aside from cycle time or broader engineering metrics) that connect engineering velocity to customer and business outcomes at a team level.
Bringing metrics and measurements in a more complete way to Remitly engineering is long overdue. It enables us to drive toward better outcomes from one of our most precious resources, our people. By understanding the friction and challenges our engineers face everyday we can improve all aspects of the software development pipeline, from tools, to code, professional development, time management, project management, and possibly more. I am personally excited to take this step and in a more quantitative light connect what engineers do everyday to customer and business value.
Check back for my next blog post as I’ll report back on progress, what we’ve learned, and where we’ll go next!
