Increase your code coverage using Istanbul
Often times, we talk about having 100% code coverage of our projects. Truth is, having 100% code coverage is a shining star on our codebase, but it is extremely hard to accomplish for an existing codebase for which coverage was not the topmost priority in past. The reason is very simple: when you do not write code in a ‘testable’ way, you cannot write unit tests for your code, which means, you cannot have 100% coverage.
The best way to get 100% coverage is to keep evaluating coverage as you develop the code. In a Test Driven Development (TDD), coverage comes along with the effort. However, in a non-TDD, you need to be very alert on calculating coverage as you go.
Before we go on, please note that I am not an enforcer of 100% coverage, but driving towards that goal usually enlightens us with lot of hidden flows in the code which we might skip testing otherwise.
Tools:
Since I have been working with NodeJS lately, I am going to talk about one of the most popular code coverage tool, istanbul. Assuming you can get the basics of Istanbul from its own documentation, I will just present some techniques to use Istanbul to increase the coverage of an existing project.
Istanbul presents some decent html reports which can be utilized to find code which is not covered.
Things to remember:
- A piece of code must be testable. If an exportable module contains a function, it should be created in a way that data can be injected from outside, so that a test can send the data and assert on the desired output. For example,
//example module index.js
module.exports = {
add: function(x, y){
return x + y;
}
}//example test
describe('Index', function(){
it('should add 2 and 3 to give 5', function(){
expect(index.add(2, 3)).to.be.equal(5);
});
});//code coverage
Statements : 100% ( 4/4 )
Branches : 100% ( 0/0 )
Functions : 100% ( 1/1 )
Lines : 100% ( 4/4 )
2. Conditional Statements: These are important for branch coverage. In the following example, the coverage is dropped because test case is not covering the conditional statements.
add2: function(x, y){
if (x && y){
return x + y;
} else {
return null;
}
}//test
it('should add 2 and 3 to give 5', function(){
expect(index.add2(2, 3)).to.be.equal(5);
});//code coverage
Statements : 85.71% ( 6/7 )
Branches : 75% ( 3/4 )
Functions : 100% ( 2/2 )
Lines : 85.71% ( 6/7 )
Fix it by adding a test case for else condition:
it('should return a value based on conditions', function(){
expect(index.add2(2, 3)).to.be.equal(5);
});
it('should return null when no value is passed', function(){
expect(index.add2()).to.be.null;
});//code coverage
Statements : 100% ( 10/10 )
Branches : 100% ( 4/4 )
Functions : 100% ( 3/3 )
Lines : 100% ( 10/10 )
Another example with an object in picture. Usually we use “short-circuit” evaluation to set default value, but it needs to be tested as well to complete coverage.
doSomething: function (y) {
var x = y || {}; // assign x as y or an empty object
return x.val;
}
//example testit('should return expected value ', function(){
expect(index.doSomething({val: 2 })).to.be.equal(2);
});
it('should return expected value ', function(){
expect(index.doSomething()).to.be.undefined;
});
//code coverage
Statements : 100% ( 12/12 )
Branches : 100% ( 6/6 )
Functions : 100% ( 4/4 )
Lines : 100% ( 12/12 )
Now a bit of complex functionality.
3. Async operations and conditions: Consider a simple example of reading a file:
readStatus: function(fileName, callback){
fs.readFile(path.join(__dirname , fileName),'utf-8',(err, data) => {
if(err) callback(err);
if (data.length > 0 ) {
return callback(null, data)
}
});
}//test
it('should read the data from the test file ', function(done){
index.readStatus('test.json', (err, data) => {
expect(data).to.equal('this is a test file')
done();
});
});//coverage
Statements : 94.44% ( 17/18 )
Branches : 80% ( 8/10 )
Functions : 100% ( 5/5 )
Lines : 100% ( 17/17 )
Notice that coverage of Statements and Branches has decreased significantly, even though we have a test. Evaluating the report created by Istanbul, you will see something like this:
Clearly, the error scenario is not covered. The ‘I’ and ‘E’ letter signify that if-else condition was not covered for the statements, hence the branch coverage is reduced. In order to fix this, lets create a test which covers the error condition.
it('should throw an error if the file is not read ', function(done){
index.readStatus('notexist', (err, data) => {
expect(err).with.deep.property('code').to.equal('ENOENT')
expect(data).to.be.undefined;
done();
});
});//coverage
Statements : 100% ( 18/18 )
Branches : 90% ( 9/10 )
Functions : 100% ( 5/5 )
Lines : 100% ( 17/17 )
So coverage above is increased for Statements but Branches is still at 90%. That is because, the else part of if(data.length > 0) is not yet tested for. Lets create a test for that. However, there is a flaw in our function under test. It is not returning anything in the callback if file is found but data is empty, ie. data.length is ≤0. So we need to fix the function first :
readStatus: function(fileName, callback){
fs.readFile(path.join(__dirname , fileName), 'utf-8', (err, data) => {
if(err) callback(err);
if (data.length > 0 ) {
return callback(null, data)
} else {
return callback(null, null)
}
});
}Now the test becomes:
it('should return undefined when file exists but no data is present ', function(done){
index.readStatus('test_empty.json', (err, data) => {
expect(err).to.be.null;
expect(data).to.be.null;
done();
});
});//coverage
Statements : 100% ( 19/19 )
Branches : 100% ( 10/10 )
Functions : 100% ( 5/5 )
Lines : 100% ( 18/18 )
Example of an API test coverage:
// a simple app
var express = require('express');
var app = express();
var people = [
{
name: 'John Doe'
},
{
name: 'Jane Doe'
},
{
name: 'Jim Doe'
}
]
app.use('/assets', express.static(__dirname + '/public'));
app.get('/people', function(req, res){
res.send(people);
});
var server = app.listen(3000);
module.exports = server;Corresponding test
var expect = require('chai').expect;
var request = require('supertest');
describe('app', function(){
var server;
beforeEach(function(){
server = require('../app');
});
afterEach(function () {
server.close();
});
it('should respond with status code 200', function(done){
request(server)
.get('/people')
.expect(200)
.end((err, res) => {
expect(res.body).to.be.an('array');
done();
});
})
})//
Statements : 100% ( 9/9 )
Branches : 100% ( 0/0 )
Functions : 100% ( 1/1 )
Lines : 100% ( 9/9 )
4. Async operation with conditions : Consider following api
app.get('/api/todos/:uname', function(req, res, next){
Todos.find({username: req.params.uname}, function(err, results){
if(err) {
next(err)
}else{
res.send(results);
}
});
});Corresponding test:
it('should respond successfully to /api/todos for a username', function(done){
request(server)
.get('/api/todos/test')
.expect(200)
.end(function(req, res) {
expect(res.body).to.be.an('array');
done();
});
});
//coverage
Statements : 97.06% ( 33/34 )
Branches : 75% ( 3/4 )
Functions : 100% ( 5/5 )
Lines : 97.06% ( 33/34 )Coverage is down because we have not yet checked for error condition.
Introducing Stubs
Stubs or mocks play very important part in unit testing. When you do not know how you can encounter an error condition in a normal scenario, you need to use a stub, basically, a mimic of original function, but which is under your control. You can force this function to throw an error the way you want.
Going back to the above scenario, we need to cover the error condition which is highlighted in pink. sinon is a popular library in Node which can be used for this purpose.
We create an object of Error and pass it on to stub as a result of the callback from find method on Todos. Notice that we are using the yields API on sinon.stub for this. yields takes the argument list that the callback should be called with.
it('should throw an error when any kind of error is encountered', function(done){
var stub = sinon.stub(Todos, 'find');
var expectedError = new Error('oops');
stub.yields(expectedError);
request(server)
.get('/api/todos/test')
.expect(function(res){
expect(res.error).to.have.deep.property('text').to.contain('oops')
})
.end(done);
});So our complete test suite looks like below:
var expect = require('chai').expect;
var request = require('supertest');
var sinon = require('sinon');
var Todos = require('../models/todomodel');
var AssertionError = require("assert").AssertionError;
describe('main app', function(){
var server;
beforeEach(function(){
server = require('../app');
});
afterEach(function () {
server.close();
});
it('should respond successfully to /api/todos for a username', function(done){
request(server)
.get('/api/todos/test')
.expect(200)
.end(function(req, res) {
expect(res.body).to.be.an('array');
done();
});
});
it('should throw an error when any kind of error is encountered', function(done){
var stub = sinon.stub(Todos, 'find');
var expectedError = new Error('oops');
stub.yields(expectedError);
request(server)
.get('/api/todos/test')
.expect(500)
.expect(function(res){
expect(res.error).to.have.deep.property('text').to.contain('oops')
})
.end(done);
});
});/coverage
Statements : 100% ( 34/34 )
Branches : 100% ( 4/4 )
Functions : 100% ( 5/5 )
Lines : 100% ( 34/34 )
Summary:
A code base with 100% coverage gives your code a shield of protection against some of the unforeseen errors. Even though some errors do not make sense, the effort of covering your code 100% makes you go through certain scenarios, which you may not otherwise. And that results in a solid software product which you can be confident about.
Useful resources:
