How we approach testing and QA at Old St Labs
Here at Old St Labs we care about quality. We believe that it is impossible to achieve quality without testing.
Testing is often an overlooked part of development. It is time consuming, tedious, and the philosophy is not always easy to grasp, so it’s often left out of the development process.
But programming without testing is a sure recipe for trouble down the line. The minutes spent testing today will be hours saved tomorrow.
It is not excessive to say that we spend one third of our time writing tests.
Without it, we would spend an unreasonable amount of time debugging our app by hand.
The way we do it
We like Behaviour Driven Development (BDD) and Test Driven Development (TDD). We try to write tests before we start programming, as much as possible.
Between the moment we approve a feature and the moment we release it, the code will go through this process:
- Acceptance testing
- Unit tests
- Code review
- Visual regression tests
- Manual testing
- Smoke tests
Acceptance testing
When we start working on a feature, we start writing user stories. They describe the functionalities of the feature in simple steps.
We use Angular to build Vizibl, and there is an amazing tool to run Acceptance Tests against it. It’s called Cucumber, and it allows us to translate Gherkin syntax into Protractor tests.
This system allows us to write easy to understand scenarios as we go through the design and plan the sprint. Each action can be re-used by the QA team without the dev team having to write more tests.
These scenarios will all fail at first, then start to pass as we release parts of the feature. The team can release the feature once all the scenarios pass.
While we write these tests at the start, we run and adapt them as we release parts of the feature.
Unit testing
Once the team has agreed to all the acceptance criterias, the actual development starts. As mentioned before, we use Angular to build our application. Each angular component, filter or service must have its set of unit tests.
The developers are in charge of writing the unit tests. The QA team is not involved in that process.
Each merge request is peer reviewed, and test coverage is an important factor of quality. Don’t expect to get away with lousy tests just because QA isn’t looking at them!
We use Jasmine to write unit tests and Karma to run them. We also use a mocha reporter because it’s prettier, and around here we like what is pretty. Although there has been discussions about using the Nyan cat reporter…
Code review
As mentioned before, a co-worker will review every bit of code, and will decide if it is good enough to merge or not. What does good enough mean? Enough tests, easy to understand algorithms, no forgotten console logs… Well.. Clean code, basically.
This is the main reason why you should bring snacks to the office every now and then. You may notice your pull requests getting merged much faster…
“If it’s not tested, it doesn’t work.”
Visual regression testing
As our app continues to grow, it becomes almost impossible to know that the CSS you just wrote didn’t break something else. This is where regression testing becomes your best friend.
We tried every possible solution we could find but none was good enough. So we decided to build our own solution, called LeRegression.
LeRegression works in several steps. It takes screenshots of your app, then compares them to a reference (the master branch). It then displays the results and shows you where there has been regressions. It can be hooked to github, and to any CI environment. For faster results we use the computing power of Browserstack to browse our app.
This tool has been a lifesaver many times and saved us countless hours of manual testing. And even with 10/10 vision in both eyes, you always miss something!
Manual testing
Because if you want something done you need to do it yourself, we do have a step involving manual testing. Our QA team will often make you grunt and swear, finding every last issue that may have slipped through the net. But we love them for it, and they may even be the most important part of all our testing process.
Manual testing is also a continuous process. The QA team is not just responsible of finding issues and reporting them. It is their responsibility to verify that new features are well implemented.
They ensure that developers fixed bugs and didn’t break anything else. They keep performing exploratory testing tasks to discover hidden issues. More importantly they guarantee that we deliver everything to the high standards.
In the end, we’re programming for humans, not machines.
Smoke testing
The very last step, the last little check to make sure everything looks fine. While manual QA of the feature in development is thorough, the smoke test is a quick overview of the product. It allows us to make sure that no major functionality broke somewhere else on our app.
You know, just to sleep on both ears.
That’s all folks!
This is not an exhaustive list of all the possible tests that a team can put in place in their development process. But it’s what we do. It ensures quality and stability as we release more and more features.
As we keep growing, we’ll add more steps to our release process. The next one may be a code analysis tool. And then who knows, stress tests? Soak tests? The sky’s the limit!
Because at Old St Labs, again, we care about quality. And if you do too, you should join us.
