Node.js Community Code Coverage — A year in review
This blog was originally posted on IBM developerWorks’ blog. It was written by Michael Dawson.
It has been just over a year since the Node.js community started capturing and publishing code coverage data on a nightly basis. I thought it would be a good time to take a look at the progress on improving coverage so far. If you want more background on the initial code coverage implementation you can go back to this earlier blog Code coverage for Node.js drives improvements.
To start I summarized the data that has been posted to coverage.nodejs.org over the last year:
As you can see we started at 89.37% JavaScript coverage and 88.3% C++ coverage and are now at 94.3% JavaScript coverage and 91.4 % C++ coverage. This is an improvement of 4.9% on the JavaScript side and 3.1% on the C++ side. Since we started off at a pretty high level I think we’ve done well.
This improvement is due to the hard work to a wide range of community members who have chipped in to add and improve tests. It is a good example of where small to medium contributions by a wide range of people can add up to good progress. A big thank you to everybody who has helped out !
Along the way support for generating the coverage results on OSX was added through pull request 16163 by Evan Lucas. That should make it easier for developers using Mac machines to check out the effect of their changes on code coverage. We now have linux and OSX support. It would be great to have Windows as well, but it is good to make progress one step at a time.
You may have noticed that it was not a smooth path from where we started to were we are now. There are a few major dips along the way.
The drop around the start of April was due to the addition of experimental support for N-API. We expected this to happen in advance and planned to work on the testing as outlined in this issue. In just over a month we had addressed the gap in testing and coverage was higher than before. The nightly coverage reports helped focus this effort and confirm when we had achieved an acceptable level.
Similarly, a drop mid July on the C++ side was most likely related to the addition of http2 and we recovered relatively quickly.
The drop around Oct 10 was due to a change that caused the Native addon tests to no longer run as part of the standard Node.js testing. This was spotted due to the drop in coverage illustrating another benefit of the nightly code coverage runs to the community. It makes sure that we catch regressions in the test suite itself !
Looking at the history for the last year, the key observation is that despite some ups and downs our general trend is upward and we continue to make progress in improving our overall coverage. I look for to further progress overs the next year.
I’m happy to have been able to work with the community on this front. If you’d like to read more about some of IBM’s other work on Node.js both internally and within the Node.js community check out our Node.js developer center.
