Quantifying Technical Debt And Ability To Scale
What is technical debt and why is it important when considering scale?
Debt, technical or otherwise, is a necessary dimension of all business and product development practice. Because debt is omnipresent, measurement of debt to determine whether it is acceptable at any given point in time is critically important. This allows you to make informed decisions about paying down debt, maintaining the current level or accumulating more.
Let’s look at the rough phases of a typical SaaS company first to understand where scale fits in and determine where technical debt is rooted:
- Idea — The founder or founders devise a hypothesis as a solution to a problem.
- Problem/Solution Fit — The hypothesis is tested, usually over the course of many iterations and a clear solution presents itself to the problem. (vast oversimplification)
- Solution/Value Proposition Fit — Occurs when the initial solution is tested with an early adopter group. The messaging and solution are adjusted to fit to prepare for the next phases.
- Product/Market Fit — A clear feedback channel is in place with the customer base and there is acceptance in the marketplace for the product. We’re now setup to scale.
- Scale — Is the iterative process of building out the business process and systems to add more and more customers to your business in an increasingly efficient manner.
Technical debt isn’t a simple measurement that can be applied to your product development. It’s not just code quality, testing coverage, speed of deployment, or the efficiency of your development process. The synthesis of leadership and managerial decisions and biases and your unique collection of good or bad habits in your product development process results in practices that are biased towards debt either being measured or not. Taking note of debt-measurement bias is especially important as your product transitions from problem/solution fit to product/market fit because this is the stage when your ability to assess debt can disappear from the radar altogether.
The transition from problem/solution to product/market is a critical phase in the life-cycle of your organization. It is a time of great flux. There usually isn’t a clear connection between idea and market fit so there are a lot of changes in direction and velocity manifesting in experimentation. While most experiments will fail, one or more will be successful, resulting in an MVP. The MVP is then expanded into a production-ready product, more experiments are performed and product market fit is achieved. Then it’s time to scale.
During transition from problem/solution to problem/market things can run amok and if debt bias is not identified, measured and understood things can get out of hand, building up in the shadows. Poorly understood and quantified debt is sure to rear its ugly head just when you are ready to scale.
Generally management tries to maintain a balance between iterating quickly on experiments and preserving a cohesive, loosely coupled, readable, and duplication free code base. At the end of the day however, you can’t have both worlds. Where do you draw the line to ensure you don’t mortgage too much of your company’s future for current speed and agility, incurring too much technical debt?
While you are hyper-focused on delivering an MVP to market, you must simultaneously weigh the debt ramifications of quick iterative delivery against the cost that will be incurred when you eventually need to move your product into scale. This is where leadership and debt-analysis comes in.
Here is a road map to measuring technical debt:
- Code coverage — This is pretty straight-forward. Most engineers are familiar with the concept of having their code written with an associated unit test attached. Pick a percentage here and stick to it. Many tools are available to facilitate and track against this.
- Code coupling and cohesiveness — This is a bit more complex to monitor, but you generally want loose coupling and high cohesiveness. This translates into code that is easier and faster to read and test.
- Cyclomatic complexity — This is a deeper level and relates to the number of paths through the application. If there are too many OR God forbid there are unknown paths thru the application this can lead to an impossibly complex application to maintain.
- Duplication of code — This relates to the amount of code that is performing roughly the same function within the application. This presents two issues. First, the more lines of code, the more possibilities for a bug to be introduced; second, the more duplication of functions means more difficulties making changes and testing.
- Refactoring and Cleanup — As development occurs and unused code is removed, to what level is the code groomed for adherence to your evolving coding standards.
There are tools out there to test most of the above but the tools won’t help you if you don’t have a general directional mandate to adhere to some level of code quality.
So how do we quantify technical debt?
I propose a quantification index tying directly back to solid metrics, stated above, along with a few qualitative measures to apply to management techniques, cultural standards, and general communication that indicate bias towards the level a company is ready to scale.
Let’s break this down a bit further:
- Quantitative measures:
- Code coverage — Percentage of code covered by unit testing.
- Coupling and cohesiveness — Metric indicating the bias towards or against the application of generally accepted code decoupling practices.
- Cyclomatic Complexity
Number of paths through the application as a ratio to the number of paths through the business process.
Percent of unknown paths through the code. — More of a subjective measure, this can manifest in the way the development group reacts to estimation of new feature development.
- Code Duplication — Duplicate methods and functions as a ratio to total methods/functions.
- Automation — Ratio of automated functions over the total number of functions in the business process.
- Product Development/Support — Ratio of effort devoted to new development vs. support of the product.
- Data Driven — Are metrics being tracked, are they being tracked and questioned regularly, and are decisions being made based on the results of these metrics.
2. Qualitative measures:
- Timeline horizon — How far out does the leadership look in regards to the product technology roadmap.\
- Commitment to Coupling — MVC, micro services, SOA. A scale of how committed the leadership team is to generally accepted coding decoupling practices. — The bias should be towards coupling, which translates into better metrics in the quantitative area above.
- Commitment to Process
Agile — Are agile practices used, and to what extent (scale 1–10) — Never perfect, but basic understanding and adherence to the more important aspects of agile product development, with time boxing and estimation being at the top of the list.
Lean — Are lean principals followed, and to what extent (scale 1–10) — How well does the team and leadership focus on solving the bottlenecks in the business process? Are they distracted with many tasks, or hyper-focused on the most critical areas that are restricting throughput in the customer workflow.
- Team Communication — Conflict vs. Collaboration. (scale 1–10) — How well does the team communicate? Are difficult questions being asked? How many team members are participating in relevant discussions or is there only one person speaking?
- Leadership Bias — Is there clear decision-making authority? (scale 1–10) Are team members provided autonomy in areas of specialization? Is accountability a company value and is it adhered to?
- Vision — How well does the team understand the product vision? (scale 1–10) — Does leadership have a clear vision, is it articulated well and frequently, do employees understand it and attach themselves and their work to that vision?
The intent is to balance speed of delivery of your MVP with an appropriate amount of accumulation of technical debt to ensure a safe path to scale. The first step to understanding debt is to be able to accurately measure it. By using the yardstick above you and your team will have clarity on your quantitative and qualitative debt levels based on your scaling needs as you transition from problem/solution to product/market fit.
Read more by Zach: zachsherry.net