Richard Clayton wrote a painful yet enlightening post called Failing at Microservices — please avoid our mistakes, where he ends it with:
Personally, I preferred the microservice architecture; we just couldn’t make it happen.
Reading Richard’s words reminded me of so many similar situations from my own history, both as a software engineer and later on as an engineering manager. Did you ever try integrating a new technology or concept into your project and just couldn’t make it to happen?
I did. It was frustrating mostly because I knew what I was doing. I had all the answers, or so I thought. Still, I couldn’t make it happen. While I can’t know for certain if the problems Richard was facing were due to skill debt, I can share some of the observations based on my experience of failing to make similar changes happen.
I believe skill debt starts when people prefer to stick to existing solutions because they don’t have faith in the possibility of applying newly gained knowledge in a beneficial way. It happens when we invest time in gaining the knowledge but neglect to build the trust along the way, both internally with our team and externally with our peers.
Increasing skill debt is an outcome of an organization where trust is deteriorating.
To avoid increasing my team’s skill debt, I started to work closely with my teammates on the methods we can use to gain organizational trust both within the team and with other teams in the organization. I’ll share these methods, including how to distribute that effort to prevent bottlenecks, but first let’s make sure we use the same vocabulary.
Defining skill debt
Skill is “the ability, coming from one’s knowledge, practice, aptitude, etc., to do something well”. That means gaining the knowledge and putting it to practice enough times to learn not only how to do it well but also when it’s the right tool for the job.
I will define “skill debt” by kindly borrowing and adjusting Martin Fowler’s definition of Technical Debt:
You have a piece of functionality that you need to add to your system. You see two ways to do it, one is based on your existing knowledge but you know that it is not the right tool for the job — you are sure that it will make further changes harder or even impossible in the future. The other results in a cleaner design, but will take longer to put in place as it requires a non trivial learning curve in addition to the work itself.
How to tell if your current skill debt is putting your team at risk?
Increasing skill debt leads to stagnation. At this point, your teammates will no longer believe that changes are possible or desired. When people do more of the same for too long, without an option to change that, it could kill any form of innovation or people growth.
“I wasn’t learning anything new, so I decided to quit” is the syndrome of such circumstances.
At some point in our professional life, we understand that building a software organization is eventually about people problems, not software problems. It starts with solving our customers’ pains and continues by nurturing an environment where software could be built while our team remains happy and highly engaged with their work.
So, watch out for these behaviors:
- Dis-empowered mindset — Your team is waiting for someone to arrive to fix the problem, e.g. an expert in some framework that you want to use. Until then, the team will avoid doing anything out of their comfort zone.
- Increasing cynicism and lack of empathy — experienced engineers will increasingly become more cynical about the team’s ability to catch up with the skill debt. You’ll slowly see camps inside the team of people who consider themselves better than the others, and the communication will reflect that. You’ll see Pair Programming made between the same people. You’ll see people taking notes during meetings only from people they respect inside the team. People will avoid teaching or mentoring other teammates, as they’ll believe it’s a waste of their time.
- Reflecting blame — the blaming game happens more often inside the team, e.g. “we have shitty code, and no one cares” — or — blaming management for lack of time investment e.g. “they do not give us time to do things the right way”.
Who should own your team’s skill debt?
Building trust is a mindset, just like feeling committed to write high quality code or delivering high quality product to our customers. As such, owning the responsibility has to be distributed. It has to become a core value of the team.
Many Engineering Managers believe that they should be responsible to own their team’s skill debt. It is even more common if the engineering manager is also the most technical person in the team. This is what I thought when I was an Engineering Manager for the first time. I see it differently today.
Managing skill debt contains many mental challenges such as keeping morale high, deciding when to invest in reducing that debt, keeping track on it so the team would feel it’s in control, getting the required support from management etc. It also includes many technical challenges to gain that knowledge, invest time in training, refactor existing code to fit a new paradigm etc.
I see two areas that the team needs to deal with managing skill debt: technical and attitude. Let’s start with the former and whom I believe should own that.
Technical Leads’ ownership –
These are the most technical people in the team. I believe that Technical Leads should spend at least 20% of their time in teaching other teammates or other teams in the organization.
We don’t need more software architects. We need more teachers and mentors to amplify our people rather than our software architecture.
What should Technical Leads monitor?
- Who’s currently becoming a bottleneck due to their unique expertise?
- Who’s currently unable to deliver certain features due to lack of knowledge or experience?
- Where is the team’s weak spot in terms of technical skills?
- During Design Reviews, has the team thought of and communicated the following:
-  What happens in case of a disaster? Is there a rollback plan?
-  What is the migration plan?
-  Are there milestones in this process? Can we create them to reduce risk?
-  Is there training missing for others to be successful at it? Can we start with that?
-  What are the timelines?
The outcome of each question is a suggestion for how to fix it going forward. It could be by conducting Code Reviews or Pair Programming. It could be by buying books for the team and do a weekly 30 minutes meeting to share what each person took from the book. It could be by looking for interesting meetups for other teammates.
It’s up to the Technical Leads to keep track on it. They should come with suggestions to the Engineering Manager on how to resolve each problem so the team could keep running in the same pace for the long term. And yes, they should ask for additional training time or resources if needed.
Engineering Managers’ ownership –
When I was an Engineering Manager and one of the Technical Leads in the team came to me saying, “our code quality is getting worse by the minute”, I liked to reply: “Why is that? What are you going to do about it?”
This was a poor answer, not because I believe it isn’t true. It shows that it wasn’t obvious for them that this is my expectation of them. It wasn’t explicit, so we each thought that the other person is going to take care of it. If this is your case, fix this.
What should Engineering Managers monitor?
- Is there a feeling in the team that things are under control?
-  Skill Debt can be an opportunity for growth, as long as it does not overwhelm the team into an analysis-paralysis state of mind.
-  In your 1:1 with the team, ask them “What is not working well today? What keeps slowing you down when working on our product? Do you feel it’s getting better in the past 3 months? If not, what can we do to fix it?”
- Are people engaged with their work? Are people happy? Is there a positive attitude even when there is a gap in knowledge?
- Are you creating learning habits for the team?
- Is there a room for everyone to speak their mind or is it owned by very few?
- Is the current skill debt or technical debt an indication for a missing hire? While it shouldn’t prevent the team from learning, sometimes this is a strong indication that you need more Technical Leads to help balance these responsibilities.
- Are other teams more comfortable working with us? Do they feel more relaxed and confident in our ability to deliver or own a big transformation? You can find out a lot by doing 1:1’s with Engineering Managers from other teams and ask for their view from the outside.
- What can you learn from other teams? Are you doing enough to ask for help?
- What can you teach other teams? Do you motivate and incentivize your teammates to share with other teams? I highly recommend reading IDEO’s Culture of Helping.
I think that Engineering Managers should work closely with the team’s Technical Leads to make sure expectations are set and they have the required resources (time, mostly) to sit with various teammates and help them improve.
Engineering Managers should pay extra attention to how people talk and react.
Our competitive advantage will always be in the way we designed our organization to work together and adjust, rather than the architecture implemented for the current solution. Reducing skill debt is about investing time in communication and building trust just as much as it is about gaining certain knowledge and experience.
I want to thank Darragh Curran for his great input, thoughts and feedback on early drafts of this post.
P.S. did you check my side-projects?
1. SoftwareLeadWeekly — a free weekly email, for busy people who care about people, culture and leadership.
2. Leading Snowflakes — The Engineering Manager Handbook: practical tools and techniques for programmers who want to lead.