The Ruby 2.5.0 feature nobody talks about

We have branch coverage now, and that’s great!

Background

Since many years the ruby community asked to have branch coverage. That’s something that if you worked a little bit with JS frameworks you really missed. If you don’t miss it, you probably don’t know what is branch coverage and therefore you are in the right place.

Line coverage

When you execute your code, you can know which lines have been executed and which ones have not been executed. Lines coverage is what has (almost) always been available in ruby. Let’s take this example:

# hello.rb (with line numbers)
1: def hello(number)
2:   if number == 1
3:    'world'
4:   else
5:     'mars'
6:   end
7: end
8: 
9: hello(1)

if we analyse the coverage by running:

Coverage.start
load 'hello.rb'
puts Coverage.result

we will obtain:

{"hello.rb"=>[1, 1, 1, nil, 0, nil, nil, nil, 1]}

which gives us the coverage for each line of our hello.rb file. Ones are for covered lines, nils are for syntax and white lines, zeros are for uncovered lines. We’ll consider ones and nils as covered (Y) and zeros as uncovered (N) for simplicity. Let’s see the result then:

Y -> 1: def hello(number)
Y -> 2:   if number == 1
Y -> 3:    'world'
Y -> 4:   else
N -> 5:     'mars'
Y -> 6:   end
Y -> 7: end
Y -> 8: 
Y -> 9: hello(1)

It correctly reports that we never executed the “mars” line and therefore that line is not covered. That’s because we executed thehello method only with parameter 1. If we would call also hello(2) we would see everything covered. Easy.

Line coverage reports which lines have been executed, and which have not been.

What happens if your code looks as follow?

# hello.rb
1: def hello(number)
2:   (number == 1) ? 'world' : 'mars'
3: end
4:
5: hello(1)

We have the exact same code but we are now using the ternary operator and therefore we have everything in one line. The coverage result will be:

{"hello.rb"=>[1, 1, nil, nil, 1]}

Y-> 1: def hello(number)
Y-> 2:   (number == 1) ? 'world' : 'mars'
Y-> 3: end
Y-> 4:
Y-> 5: hello(1)

That’s the first issue with line coverage. Line coverage works on a “per-line level” and is absolutely not capable of understanding that we never went into the “mars” branch.

With branches we mean the two possible execution flows that and if block can have: they are the then and the else branches.

In our case the then branch is the one that returns “world”, the else branch is the one returning “mars”.

Our line coverage works great, but nothing tells us that the “mars” branch has never been executed. We clearly miss branch coverage.

Branch coverage

ruby 2.5.0 introduces branch coverage! That’s a great news because in the future, tools like simplecov will be able to report us exactly which branches of our code have been executed and which have not been.

Code coverage is mainly used in test suites, to report how much of the application code has been covered by tests. This means more precise numbers in the future and the assurance that all the branches of your code have been covered. That’s great!

In our example above, if we use ruby 2.5.0 and run

Coverage.start(branches: true)
load 'hello.rb'
puts Coverage.result

we will obtain:

"hello.rb"=> {
  :branches=> {
    [:if, 0, 2, 2, 2, 34]=> {
      [:then, 1, 2, 18, 2, 25]=>1, 
      [:else, 2, 2, 28, 2, 34]=>0
    }
  }
}

which reports for each branch (then and else) their coverage. Each key is in the format

[clause, id, row_start, col_start, row_end, col_end] => covered 

and we can see that we have a else branch uncovered (0).

The hidden branch

Another example were the line coverage wouldn’t be enough is the following:

# hello.rb (with line numbers)
1: def hello(number)
2:   if number == 1
3:    'world'
4:   end
5: end
6: 
7: hello(1)

Can you guess why?

Line coverage would simply tell you that everything is covered and does not see that we never executed our code with a number different from 1.

hello(1) # returns “world”
hello(2) # returns nil

The new branch coverage will, instead, see perfectly theif block and give us the coverage for both branches:

"hello.rb=> {
  :branches=> {
    [:if, 0, 2, 2, 4, 5]=> {
      [:then, 1, 3, 4, 3, 11]=>1, 
      [:else, 2, 2, 2, 4, 5]=>0
    }    
  }
} 

and tell us: “hey! you executed the hello method passing 1 but, what would happen if you pass something different?”

Conclusions

That’s the greatest feature ruby 2.5.0 introduced and it will still take some months before seeing a tool using it, but the ruby community has now branch coverage and that’s a great news.

Having a test suite with branch coverage 100% guarantees that you really consider all possible cases and execution flows of your code and reduces the possibility of bugs.

At Renuo we are all looking forward to integrate that in our test suites to increase the quality of our software even more. We always guaranteed 100% lines coverage with tests and this will allow us to push towards 100% branch coverage also for our ruby code! 🙌