Who manages technical debt levels?
It’s critically important for engineering groups adopting modern agile to set a clear distinction between user stories, that is, specifications of new business value that are prioritized by product managers, and development chores, tasks specified and prioritized by the engineering team itself.
This last point is crucial: the engineering team decides when it does chores, not product managers.
The reason this rule is so important is because chores include paying off technical debt, aka refactoring. When the engineering team forsakes the privilege of refactoring at their discretion (as so often seems to happen in Scrum) you can be sure that technical debt will spiral out of control. At worst, that debt will reach levels that crush creativity and productivity, and force large rewrites.
Back in the 90s, when banks would send you non-stop balance-transfer offers. a friend of mine from church figured out that he could borrow a large sum of money “for free” by continually flipping his existing credit card debt onto a new zero-interest card every month. (Same guy that convinced me to work with him at a boiler room operation.) The free money scheme worked for awhile, but the size of the debt and complexity involved in managing payment got bigger each month. More on that later…
Giving product managers control of technical debt level is kind of like juggling credit cards. New expenditures always take priority over paying down the principal. Eventually it’s going to end badly.
SUSTAINABLE PACE IMPLIES SLACK
No, I’m not talking about the office group chat software.
Slack is the portion of time used to work on stuff other than user stories. It fluctuates from 10–30% of the available hours in the week.
Developer chores and refactoring happen during slack time. I also want my teams using slack time on self-directed learning activities, experimentation with new languages technologies and approaches, reading and writing blog posts, and contributing to open source.
In bigger companies, there’s also the option of moonlighting as a guest pair on other teams — representing an opportunity to help out the greater cause, but also knowledge sharing and spreading of wisdom across departmental silos.
DISCRETION
One of my practical problems with junior engineers is that most don’t know how to properly give themselves the right amount of slack. Some will waste time on lost causes or shiny objects. Others expect to be told exactly what to do at all times.
Knowing how to properly balance time requires discretion based on real experience. Often a degree of luck and good timing is involved, because often the opportunity to pay off technical debt pops up at an unexpected moment. You might be in the middle of coding a feature when inspiration hits. Junior people don’t generally know how to evaluate the tradeoffs involved in taking the productivity hit now versus later. One of the myriad reasons I love pair programming is because two heads are better able to make those kinds of judgment calls.
If you can’t have mostly senior people on your teams, then make sure that all include at least one confident and professional senior engineer that you trust to have the necessary discretion regarding sustainable pace and slack time.
VELOCITY VS. VOLATILITY
Velocity denotes the pace at which a team is delivering business value. Slack time eats into velocity. You don’t run a team at its (theoretical) full velocity all the time for the same kinds of reasons you don’t drive your car around with the engine revved to the redline. It would burn out.
Having the proper amount of slack time is similar to saying that the team is working at a sustainable pace, one that allows enough time for addressing technical debt on a regular basis. The most important indicator of a team working at sustainable pace is low volatility. The velocity of the team should not swing wildly from week to week. It should be stable and lend itself to accurate predictions about delivery expectations.
NEGOTIATION
When balance is lost, and a team has not been giving itself enough slack, there will undoubtedly be some negotiation involved in getting things back to their correct state of affairs. Every team has external dependencies. Product managers may have made unreasonable promises to their stakeholders. Deadlines may be in place.
Negotiation does not simply mean agreeing to work harder, longer hours or cutting corners. The team should never agree to drop its core practices with the pretense of moving faster. A confident, experienced team lead knows how to stand firm against this kind of pressure. Expectations must be adjusted, as early as possible.
The primary tool for adjusting expectations is scope management. Assuming the team has done a good job of forcing stories to be maximally decomposed and of roughly equal size, then adjustment of expectations can begin by moving some stories out of scope for the milestone that is in danger of missing its deadline. The conversations that lead to movement of scope take the form of negotiations.
If user stories have not been decomposed sufficiently, then a round of analysis might be in order prior to negotiations. The sooner this happens, the better, which is why I think story grooming should never be a standalone activity. Someone with product management responsibility should be grooming the backlog on an ongoing basis.
Scope negotiations should not be seen as a negative thing, or something that happens only as a result of broken process or promises. They happen on a regular basis between product management and a representative of the engineering team as a normal part of their healthy relationship. The team tries to get as close to an ideal state, free of technical debt and defects, as it can without blowing the product manager’s project budget. It’s a partnership. The two sides must treat each other as professionals, with mutual respect.
Professionals don’t have bosses. Professionals are partners with business. They may be employed, but they are not laborers to be managed by foremen. Rather they are experts who know how best to achieve what their employers need. You hire doctors and lawyers, but you don’t manage them, you aren’t their boss. Imagine trying to boss your doctor as he performs open-heart surgery on you. The concept is absurd. That’s the bar I want programmers to reach for. I want them to behave like lawyers, like doctors, like real professionals; not like laborers. — Robert Martin, author of The Clean Coder
Beyond delivering defect-free code and keeping technical debt to a minimum, you want your engineers to be able to drive negotiations of scope based on domain expertise. Ideally, the engineering team gets senior enough, not just technically, but in the business of the company, that they’re able to contribute to analysis of business value based on metrics and key results. They’ll want to put those measures and gauges in place, so that they can answer for themselves whether they’re providing business value or not. I consider this the ultimate level of software engineering advancement.
The line of code you don’t write is the line of code you never have to debug. — Steve Jobs
PUSHBACK
I’ve often said that one of the most important skills for an engineer is the ability to say no. Sometimes, as we just discussed, you have proof that you’re being asked to work on something of dubious or worthless business value. Other times the work you’re being asked to do is not possible to accomplish in the timeframe specified, without sacrificing your standards. Or you might be pushed to work late hours, when your mind is tired and you know that you will be introducing more bugs than functionality. It’s time to say, “sorry, I’m going home for the night. This fix can wait for tomorrow.”
In the world of mechanical or civic engineering, the ability to say no when being pushed to approve project work containing unrealistic expectations can become a matter of life or death. Bridges that are incorrectly engineered can fall down. Highways that are improperly engineered can have elevated traffic fatality rates.
However in software engineering, life and death outcomes are rarely the case. As an industry we lack discipline to the extent that some people think software developers should not call themselves engineers. They say that engineering implies a formal system of ethics, and we don’t have one of those. There are other reasons too, but I consider the ethical argument pretty compelling.
ETHICS OF A SOFTWARE DEVELOPMENT
Instead of lives lost, the cost of software engineers acting unprofessionally is wasted money. Short of complete project failure, that cost (usually not even calculated or recognized) does not come out of the pocket of the software people directly, but rather from the shareholders. In large corporations, the rank and file engineers are generally too far removed from the shareholders to give a damn about their profits. Incorporating equity into comp packages for engineers helps to mitigate that concern, but I wonder about the effectiveness.
Still, violation of ethics generally leads to a reckoning. My credit-card abusing friend eventually had his scheme collapse and was forced into bankruptcy. Many software projects end up so dysfunctional that the only reasonable choice is to throw everything away and start all over again.
Having to rewrite a system that has become mired in technical debt is so common that it doesn’t even strike people as that much of a problem. After all, the second or third version will be superior since you know the domain so much better by that point, right?
Sure, you probably will, but at what cost? Mind you, I’ve referenced the sunk-cost fallacy many times when advocating for rewrites. But on the flip-side, I have participated in enough successful projects in my life to tell you that the positive alternative — maintaining a defect-free system, at a sustainable pace, with the correct level of automated test coverage, and continuous refactoring to improve architectural facets — is achievable and preferable.
Socrates is the hero and patron saint of those of us that push for modern agility and software craftsmanship in face of corporate obstacles. The legendary philosopher was a social and moral critic of the corrupt Athenians. Like Socrates, we’re usually in the minority, and often lack the power to effect sustained changes. Too often we face execution (getting fired) as thanks for our good efforts to make things better.
Socrates questioned the collective notion of “might makes right” that he felt was common in Greece during this period… he irritated people with considerations of justice and the pursuit of goodness. His attempts to improve the Athenians’ sense of justice may have been the cause of his execution.
Luckily, the job market for people that know what they’re doing is white hot. Getting fired for wanting to do the right thing is a great way to build up a strong reputation and move on to greener pastures and more money.
If you can’t change your company, change your company.
