Setting up Ruby, Codecov and Github Actions
You write a piece of code, and you know it works. You just wrote it. You just ran it…Six months pass. It is time to refactor your code. You would kill for some tests. — Swizec Teller, You don’t need tests
Introduction
Tests are important, they give developers the confidence to change the codebase and they reduce the friction of delivering new features. They make the codebase maintainable and easy to change which is really important in the long run. They are also great for documentation and help onboarding new members to the team.
I joined Talent Protocol and one of my personal missions was to increase the code coverage in our main application.
Plan
Missions are only achievable if you have a plan! And here is my plan:
- Start adding tests incrementally
- New code should always be covered by tests (Codecov will help here)
- Have small increments of code coverage each month with defined percentage targets
- Have a way to retrieve metrics (Codecov will help here)
As I write this the code coverage of our app sits at 17%. This is low! We should aim to have at least 90% of our code covered by tests.
The goal is to achieve 90% coverage by the end of the year!
Execution
How it started?
We started by using the SimpleCov gem together with a Github action that generates code coverage reports on all pull requests. As you can seen in the screenshot the report was pretty simple. It tells the code coverage percentage and it shows the threshold. If a pull request introduces new code and the coverage percentage is reduced below 6% the Github action workflow will fail.
You can check the Github action that led to the output above here.
Soon we faced an issue with this approach. A Github action running on a pull request event submitted by a fork can’t download artifacts stored in the Github workflow. This means there’s no way to generate the coverage report and upload it to the pull request.
Since our repo is public and we welcome and incentivize contributions from our community this was an issue for us.
There’s a way to run Github workflows triggered by fork repositories. Github provides this with the pull_request_target event, but this event runs on the base branch of the pull request and we want to run it on the feature branch.
Retrieved from Github docs:
Note: Workflows triggered by
pull_request_targetevents are run in the context of the base branch. Since the base branch is considered trusted, workflows triggered by these events will always run, regardless of approval settings.
How do we solve it?
Since our repo is public we have free access to Codecov.
Instead of uploading the coverage artifacts into Github workflows we now upload them to Codecov which will then compute the coverage report and upload it in the pull request.
The report above has more information compared with our first iteration. It now tells us how much we improved compared with the base branch and it also give us information about hits and misses. You can learn more about this here.
You can check the Github action setup here.
I hope this post was helpful to you. I welcome your comments on this, and would love if you can actually use it by opening a pull request in Talent Protocol main app. We have paid bounties waiting for you. Check them here!
