Measuring project health, community edition

Developers often intuitively assess the health of open source dependencies they decide to take on. What can we learn from their practice?

Last week, I posted about measuring the health of open source projects you want to take on as a dependency. A mailing list reader replied that they really like the simplicity of just focusing on the date of the last commit (shared with permission). (If you’re not on my mailing list yet, here’s where to sign-up.)

While I myself rely on that when picking open source projects, I realized it’s not the only signal I use.

I use node.js quite a bit for software development. I must say npm, node.js’s package manager, has made me pretty spoiled. It provides a lot of data about package usage. So one of the strong signals I use, especially when I need to pick between different projects that serve the same purpose, is the number of monthly downloads of the package.

It’s usually very clear from package download numbers which project has been adopted by the community. Not only is that a guarantee of fewer bugs—if we’re to believe Linus’s Law​ that more eyes on the code improves code quality—but it also signals a much larger pool of users with vested interest for the project to survive maintainer defection.

Curious of what others did on this topic, I started a Twitter poll:

The responses so far have been quite interesting.

There’s a small percentage of respondents that don’t do anything and hope for the best. That’s a quite concerning strategy​. On the other hand, a majority of developers check the date of the last commit, as was discussed last week. A lot of people replied with their own tactics to assess project health. While I hadn’t thought about some of them at all, I realized that I was intuitively relying on some of the others, but wasn’t doing so consistently at all.

When the poll closes, I plan to collect this data to create a quick project health assessment checklist anyone can use to mitigate the risk of adopting a dependency whose maintainers’ commitment (pun intended) may very well dwindle in the near future.

Stay tuned.