Automation journey: from zero to Robot

Bynder proposed a big challenge to me: leading the Automation Project.

At that time there were already many automation tools available. That's why we came out with 3 main requirements that the tool of choice should offer:

• We wanted both technical and non-technical people to be able to write test cases in natural language using BDD scenarios. That's why we wanted a gherkin-based solution.
• Open-source browser communication framework. We just love open souce :D
• Use Python to implement automated steps. The main reason for that was that Bynder backend is also using python, meaning we could have office support from experienced byndies.

Winner tools

Behave and Selenium won the first round. Both choices were easy, since Behave is the natural BDD interpreter for Python (like Cucumber for Ruby, or JBehave for Java). About Selenium, we chose it because it is the most widely used open-source framework to automate browser actions.

Page Object

After having a bunch of functionalities automated we realised there was no control on how our code accessed the browser web elements. That's why we introduced the Page Object Pattern into our code to isolate the html locators from the step implementation.

That way, when the Bynder web frontend would release changes on any html elements, we would not go through all our automation code to change their locators. Instead, changing the affected elements once would make the code run again with less hassle.

Isn't Jenkins the greatest servant of all (tech-)times?

We’re requested to schedule out tests to run against the new release environment every night. Since DevOps was already using Jenkins to automate processes, it was a natural choice to create a new job in Jenkins that would run all the tests.

Going (pseudo)parallel

Time is precious in automation. We felt enthusiastic automating large transfers, uploading +60 different file types, performing multiple asset downloads, zip checks and swallowing full collections until we realised our tests would need more than 4 hours to execute.

Since our tests were running sequentially, for instance we would not start the +60 uploads until a certain 6GB transfer would finish. The solution looked easy: go parallel. After all, why not uploading files at the same time we would do the big transfer?

Unfortunately, Behave does not offer native parallel execution of testcases. The solution we came up was to use Jenkins in a more clever way: split tests by functionality and execute them on separated target environments to avoid test dependencies. The Selenium Grid was very useful here to allow several browsers running at the same time inside the same machine.

Tag your life

Execution time dropped, we added more scenarios to the code, so it raised again. Running the whole thing would take around 2 hours. That was fine for a full regression testing of the Bynder app. But what about having a reduced version of the execution to test the most basic stuff in less than 20 mins?

The solution was tagging. Adding tags to the BDD scenarios allowed us later on to run a subset of them. For instance, a "develop" tag would be used in only 1 of the 60+ upload scenarios. This way we tested our own automation code and execution time would be dramatically reduced!

The Robot way

I always loved robots. Unfortunately, only that would not justify dropping Behave and start using the Robot Framework. Here are the reasons why we moved to Robot:

• Robot uses Keywords to implement steps. They can be implemented in python (like Behave) or just using other Keywords, which allows way more flexibility and reusability of code.
• It has a wider online community than Behave.
• Still compatible with BDD.
• Can use Selenium2 lib.
• The Robot pabot library enhances Robot with test suite parallel execution.

Learn (Behave) Robot in 30 seconds

(Behave) Robot is based in (feature files) test suites, which contain (scenarios) test cases made out of (steps) Keywords that are implemented using (python) other Keywords or python.

Can Robot use the Page Object?

Yes. Robot can easily pass the browser instance reference to any python script. This is how we did that at Bynder:

The created global variable ${DRIVER} can be used anywhere in Robot after the browser instance is created.

See it in action on a classic Login Keyword:

Username and password inputs and click Keywords are implemented on Python code that uses Page Object at will:

As a bonus, we are able to call keyword implementations passing a "driver" that gets shifted to the "page" object through the decorator:

Et Voilà :)

Evolution rule

The fact that we moved from Behave to Robot does not mean that we will stick with that automat forever. The Tech world evolves rapidly, and what is a good solution today could be obsolete tomorrow.

Our only rule is that there are no permanent rules. It's the Queens Of The Stone Age approach: Go With The Flow.