Web testing quickstart with Cucumber, Ruby and Selenium
Remember that time you sat down and thought “before I release I will just run through these fifty or so pages in our app to make sure everything runs ok”?
No? Well, neither do I.
Manual testing is tedious and I would rather not do it. But there is real merit to it. Things break all the time and when we expect it the least.
We can’t, nor should we, avoid manual testing. I don’t even think manual testing at the core is boring it is the repetition of tests that is boring.
If we can take the repetition out of testing by using automation, we can get the same (if not higher) degree of confidence with none of the boredom or time loss.
Automated testing, even of user interfaces, has been around for a long time. Selenium was created in 2004 and became the gold standard for web testing.
Today I want to show a practical example of how we can use Selenium to automate tests.
How does Cucumber help?
If anyone asked me, why would you use Cucumber?
Using Cucumber forces you to write BDD (behaviour driven design) tests. This means that your tests don’t test code but user interactions. It forces you to test things the way a user does. Plus, they read really easily.
I opened cucumber.io expecting a similar description and I suppose the first paragraph begins the right way:
I would say that’s over-simplified but they are on to something. I believe that by writing a test first, the way you would explain it to someone in simple terms, you get high-quality tests. Also, you can even run them manually first to make sure they work in practice.
The BDD style tests even work as documentation of how the system is intended to work. I think the way they read explains the value.
What I think cucumber.io’s explanation over-simplifies is that writing a Cucumber spec is easy but implementing the test is still as hard.
A cucumber test actually does nothing. It is just a text file. That text file is then interpreted and matched with implementations of these text lines to run pieces of reusable code.
You have to implement each of those code pieces. However, organising the code this way does help with reusability, readability and maintainability.
Is cucumber still alive?
If any of you have been around for a while, you know that Cucumber has as well. The first commit was made in April 2008, that’s a whopping 13 years ago (from today's count). That’s my entire professional coding career.
But age isn’t a bad thing, look at Git, it’s three years older and more alive than ever.
But, not all products are Git. So how about Cucumber, is it still worth relying on?
I would answer: Definitely. Cucumber is a DSL (Domain Specific Language) to help make writing (or rather, reading) tests easier. As such, it’s timeless in my mind. It’s backed by a number of implementations, you can use it with most of the popular languages and how you actually execute your test, well that too is up to you.
You don’t have to use Selenium. For UI automation, I think we will see some newer tools come eventually to replace Selenium which might make it even easier.
But using Cucumber can still be worthwhile, you can even see it this way. Write your tests in Cucumber and that faithful day Selenium gives way to a new automation framework you can re-implement your step definitions and all should still work fine!
Finally, Cucumber got acquired by Smartbear 2019, one of the world leaders in test automation software. They themselves crown it as the leader in BDD frameworks.
the most widely adopted leader in the behavior-driven development (BDD) community and creator of the open source test automation framework
So Cucumber is far from dead. In fact, Google Trends show no big signs of decline although it is getting a little less busy.
You can also check out their Ruby github repo, though very stable it still has weekly commits!
How do I build a test with Cucumber?
I have created a very simple example repository to show how Cucumber tests can be set up.
GitHub - ddikman/cucumber-web-tests: This project is a super tiny start at using Capybara &…
This project is a super tiny start at using Capybara & Cucumber with Ruby to write easily maintainable tests. - GitHub…
Make sure you’ve got Ruby and RVM installed first then run this to get started:
git clone email@example.com:ddikman/cucumber-web-tests.git
rvm install 2.6.0
cucumber features command will run cucumber on the
features folder you will find
google-search.feature which defines two tests to run on the Google search engine. Their content should be fairly easy to read and understand:
The magic of this file is in the step definitions. You can find these in
features/step_definitions which is where cucumber finds the implementation of how to execute the tests. In these files, each
given|when|then are matched using regex, such as here:
One last thing to mention though, the Selenium environment and Capybara wrapper is configured in
env.rb which is run on start.
Using Capybara cuts out a lot of the Selenium setup code but it works perfectly fine to use pure Selenium without Capybara as well.
How to get started
Fork my example repo and get running. Add another test just in Cucumber syntax then run
cucumber -d to generate the missing definitions.
Add new step definitions and run the tests. Done.
Get the value by running tests, always
To get real value out of the tests, they need to run frequently. Every time you change anything preferably.
To do this manually is easy to forget, tedious and costs time. Instead, use Github Actions, Gitlab CI, CodeMagic, CircleCI or any of the other great build systems to trigger tests automatically when you commit changes.
I have used all of these and they will all do the job. They all have pros and cons but I would recommend going for whatever has the most support for what you are trying to do, is available for free and is close to where you host your code.
The hardest thing isn’t actually to trigger the tests but to run them on the right device or so. There are lots of services to handle this as well. I have used Saucelabs in the past but here are a bunch of other more modern ones:
How to run Cucumber in Github Actions
The example repository is also set up to showcase automated test runs on commit. Check out the github ci file to see how this work. To make sure we can easily run the tests visually for troubleshooting locally, I use this line in the
env.rb file to determine if we are running in CI or locally and switch headless mode:
driver = :chrome_driver
if ENV['HEADLESS'] = 'true'
driver = :chrome_headless
print("Running with headless driver")
(If you are a Gitlab fan, check out how to get your test coverage on merge requests in Gitlab for comparison)
One test is better than no test
I would love to say that it’s easy to get started and that you should. However, I can only say that you should, not that it is easy.
My hope with this article is to take the unknown out of at least one step, the writing and running of the test. I urge you to fork my repo, modify it for your own application and run it. That way you will see that writing the first test is easy.
What makes setting this up hard is finding a runner for your test (if you need a specific browser etc) and adding it to your CI, likely after your deployment has been done.
Still, having even one test will give you that added confidence that, at least, you haven’t broken the top page or the checkout flow or whatever you may choose. A single, or a few, simple smoke tests can really help speed up your workflow and remove that feeling of uncertainty after every deployment.
Then from there, you can expand. User interface tests should only be the tip of the testing iceberg so don’t focus too much on getting everything tested but try to have something in place and you will sleep better.
I hope you give it a try and give me a shout in the comments if you have other experiences, tips or tricks to get running with UI tests in your CI flow!
Thank you for reading.