Can We Measure Software Developer Productivity?
Short answer: I don’t know.
Short answer: I don’t know.
Before you move on though, I’d like to say that “I don’t know” here means that I know from personal experience that trying to measure a software developer’s productivity rarely (or never) ends well. However, I also acknowledge that, historically, there have been valiant efforts that worked to some extent, and there are likewise promising ideas today that are making the rounds towards addressing the problem.
I’ll run down most of these things here, and I do hope this helps someone out there find a good way to measure productivity (or any other indicators, for that matter) in a way that makes sense in their context.
The Holy Grail
Finding a good way to measure productivity for software developers has always been a “holy grail” of sorts. Many have tried, few have succeeded. And the reason for this is much more obvious these days than it was when software development (or “computer programming”, as they used to call it) was a young field.
Here’s a relevant quote from jazz legend, Miles Davis, on what makes a great jazz tune.
“It’s not the notes you play, it’s the notes you don’t play.” - Miles Davis
This same message holds true for software code. In its early days, software development used to be considered “production” work, not much different from building a house or manufacturing hardware. This line of thought still exists today with most non-technical stakeholders and even with some who have been working in this field for a while. More and more people, however, are starting to realize that software is not exactly construction work, but is, in fact, closer to a pursuit of creativity.
In this sense, just like notes in a tune, the code you don’t write is just as important as the code you write.
Sorry, what?
If that’s a little bit too abstract, let me explain a few ways this manifests:
- Long code. This is often an indicator of code that can be improved. If you can do the exact same thing with fewer lines of code, you can save on binary space and perhaps your own time. (Be careful not to compromise on readability though.)
- Reinventing the wheel. “There’s just too much software to write for one company to write it.” This is why open source is a thing. If someone has already done it before and it works, you should reuse that as much as you can. If you can use it as a black box library (as opposed to copying the code), that’s a few less lines of code for you to maintain in the future.
- Cognitive load. The most satisfying code is often that which took you hours to think about and a few minutes to type out. You might be able to write entire classes effortlessly if it’s something you’ve done a million times. But writing an efficient 5-line algorithm for something alien to you might take just as much, if not more, of your time.
In addition, writing more code actually means a higher risk of inadvertently writing a bug somewhere. Likewise, if your code is long enough that other people don’t bother reading it, it’s probably worth considering just abstracting it altogether.
In short, measuring “volume of work” to understand productivity just doesn’t make sense anymore, since volume in any tangible form hardly correlates to effort and quality.
So basically “no”?
The industrialist in me doesn’t want to admit that there’s no way to measure software developer productivity. As much as I believe that creativity is important in software, I’d also love to be able to tell a customer that we’re halfway done and actually mean it.
Historically, measuring productivity for software developers has been a long and arduous road — from measuring lines of code (definitely rewarding the wrong things) to function points (overly conceptual and complicated) to velocity (not exactly a measure of productivity, but a decent basis for estimations). Each one works in certain contexts, but as software evolved (and it evolved fast), some eventually became irrelevant.
However, just the fact that people have kept coming up with new ways to measure it says a lot about just how important it is to a lot of people.
The future of code metrics
There seems to be some hope. Companies like GitPrime and Waydev are coming up with new Git-based metrics that might make more sense in the way we do software work today. For example:
- Code churn means the volume of code that has been modified less than 30 days from its first commit. This indicates that we might be doing a lot of work that’s volatile and essentially make very little impact on a business level.
- Review coverage is, in the spirit of collaboration and collective learning, a measure of how much code each developer reviewed.
- Impact is, well, something I don’t fully understand. According to the documentation, it should correlate to “cognitive load”, which means how much thought might have gone into coming up with a certain commit (regardless of whether it’s an edit, an addition, or a removal). Disclaimer: There’s probably a lot more to it that I’m not fully grasping.
These and other metrics, in my limited experience with these tools, do seem very promising. I think the naming, particularly “impact”, could be better. Because whatever these metrics indicate, they definitely don’t indicate anything particularly good or bad. They might, however, represent certain trends and behaviors that we can choose to look further into.
I guess what I’m saying is, maybe we should be focusing less on “metrics” and more on “indicators”. What we can easily get is data, not information. Data is just a bunch of numbers. Information, on the other hand, is something that takes human processing to figure out. Whatever the numbers seem to indicate, we need to interpret them in the context of our actual work and our organization. Most importantly, we definitely should stop making these metrics a competition.
If you liked this post, sign up for my mailing list to get updates in your inbox. Originally published at https://techmanagement.life on March 10, 2020.
