Test Automation Day Rotterdam
Last week I went to the Test Automation Day in Rotterdam. It was all last minute but when I saw that Angie Jones and Dan Cuellar were speakers, I jumped into a plane and went.
Organisation, venue and the city
Test automation day is a small conference with around 200 people. I was surprised to see that there was almost no advertising stands from the sponsors (the only conference I’ve been before was the selenium conference in India back in 2016 and a one day conference in Barcelona, both had more sponsor booths). I’m sad to leave the conference without any t-shirt or stickers :(
On a positive note, the size of the conference makes for a much more intimate and interesting series of workshop. That still left us with a wide choice of tracks. One thing the organisation did well was to separate the talks by themes:
The venue
The World Trade Center was really nice, located in the center of Rotterdam with plenty of places to eat and hotels around.
The workshop rooms were comfortable and roomy as well.
The audiovisuals team was perfect, I can’t remember any conference I’ve attended with zero AV issues.
The content — workshops and speakers
Bas Dijkstra — Enhancing your automation efforts with Cucumber / SpecFlow
I expected this to be a session about how to apply BDD/Gherkin to my project or Pros VS Cons of using BDD. Instead, we were greeted by Bas with a “please close your laptops, you won’t be needing them in this workshop” welcome.
It was soon clear that this would be a discussion workshop, rather than a code-based one. It was actually a pretty nice session!
Bas illustrated his points with exercises of good/bad BDD scenarios and discussions with each team on how to improve them. I liked the approach because I have worked with BDD before, but it was probably a surprise for those looking to learn about it.
I gave this feedback to Bas during the conference and he said he’s going to work on the presentation to give a small tutorial on what’s BDD and how to use it.
One thing I noticed that a lot of people were confused about, they thought Cucumber was an automation tool like Selenium.
They failed to realise that BDD is just a layer you add to your tests (API tests, Unit Tests, UI tests) in order to increase both readability and communication with your team.
Although you can use it by yourself (Angie Jones wrote a blog about it), one very important point of the workshop was how to use BDD to communicate with the whole team (product, dev and QA).
All in all, it was a very nice workshop. Plenty of information on BDD, techniques and tools to use, and discussions on the pros and cons of using it.
I want to finish this one with a shout out to Bas for being a all around nice guy and for maintaining a good environment during the class with his jokes :D
Here are my raw notes from the workshop
Jon Hare-Winton — Getting automation right from the start
Jon was also a star, always interacting with attendees and trying to make us talk and discuss. As with Bas workshop, no laptops were hurt during this one. Another nice surprise.
On this workshop we discussed more about the approach to create an automation framework in your project (or if you actually need one) and what tools and technologies to use.
Similarly to Bas, Jon presented us with 3 scenarios of past experiences and we needed to discuss the approach taking into account the current team, size of the company, urgency of the project, budget for the automation project, etc…
It was funny how all (or most) of the scenarios Jon presented were past failures. It is very important to try and fail, but only if you learn from your failures/mistakes. In Jon’s case it gave him the tools to be an expert in his field leading the testing team of a major newspaper running 3000 tests per day in production…yeah, 3000 :D
This workshop was particularly interesting to me because I’m starting my own automation framework and I’m still deciding on the approach. Also, since we’re a small team, I need to take into account the developer’s experience with tests and automation and if I’m going to use the same codebase as them or a totally separate technology.
I also need to take into account that the QA’s I’m hiring are going to be very junior so I need to start by building the framework in a simple way so they can understand it and can write tests for it.
Angie Jones — Automating the Unknown: The Reality of Testing in an Artificial World
Right from the start you can tell Angie is a pro. She is totally at ease on stage, and in complete control of the flow and speed of the talk. She is funny, and interacts well with the audience, even on a full house like this one.
She talked about testing a AI/ML based project:
- what she struggled with
- how to make it visible to the product people when something is off and everyone is rushing to deploy
- how you need to stand your ground (that testers instinct, you can’t beat that!!!).
As Bas did in his workshop, she stressed the importance of APIs and databases to help automated tests, making them more reliable and fast
Everyone is talking about Artificial Intelligence and Machine Learning nowadays and saying that computers will steal our jobs: that’s a load of crap. Computers can only do what you tell them to do, who’s going to think about that crazy test that just 1% of your users will ever think about?? Yes, you!!!
You’re the one who needs the computers for testing: if you have to do 5 steps on a web app 2000 times, for sure you will use that computer power and speed…
One thing she mentioned that made everybody laugh:
- THE P.O.C.S. METHOD (™by Angie Jones): Plain Old Common Sense
Basically, when something is very complicated and is messing with your head, just go back to basics, go home, take a nap or have a good night’s sleep and use common sense, you’ll see that everything will be just fine.
If it isn’t, well, time to jump on LinkedIn and see what kind of offers exist out there! :D
I’ve been following Angie on twitter for a while, she’s a superstar in the testing world and for a very good reason: the amount of knowledge inside that brain is amazing as is her ability to transmit that knowledge and break it down so you understand it. She was one of the reasons why I went to this conference and she did not disappoint at all.
Ilari Henrik Aegerter — Automate your Automation — When Will the Robots Take Over Human Sense-Making
This was a very weird talk, funny, but weird.
Ilari is a very interesting character, coming from the Linguistics and Sociology field, for some reason he ended up working in software testing and helping out the community by creating associations and standard setting organisations.
You can tell that he really likes languages, since most of his talk was about languages and font types instead of software testing :D
This book was one of his recommendations, I’ve seen the author’s TED talks and read some of his blog posts, it is very interesting indeed.
Ilari has an interesting take on the “Robots and Computers are going to take over the world” thing:
- Option A: we all die
- Option B: we all live happily without working because robots are doing everything for us
I really really really like option B….
Not much more to say about this talk: interesting, weird, funny… You always learn something from this kind of talks when the speaker is as good as Ilari.
Fabian Basciani — Testing Event-Driven Platforms for Trading Automation
Fabian is a super smart, super intense developer, but his pace was incredibly fast and comes out a bit too “know it all”.
The system he was describing looks amazing but he was explaining it like we were one of his colleagues, like we understood all the ins and outs of a trading platform.
He has a fully tested record and playback transaction system that allows him to, in an instant, go back in time and replay whatever transaction was done based on the logs he is saving from all the ongoing transactions.
At the same time he can debug each transaction going through the system based on the same logs.
One thing he mention that I really believe as well: if you have tests covering the whole business logic you have:
- Confidence that if you run the tests the new code is ready to go to production
- If a test fails, everyone stops and fixes the issue, since every test is covering one important business logic part
- Specially helpful for new colleagues because they can play around with the code without much danger and they learn the logic of the code by checking the tests
Use simple thread to avoid concurrency issues
My two cents: in a system as complex and heavy as a trading platform, you should use the full power of a computer, and that includes multithreading and concurrency. That is something you need to deal with, not avoid it because it’s hard to do.
Yes, it is a P.I.T.A to build such a system, but I think the advantages clearly outweigh the disadvantages.
I guess it all comes down to how much processing power you have at your disposal, nowadays processing power is not that expensive using the right cloud services.
Amy Phillips — Test Automation: May contain side-effects
This talk was more related to continuous integration and continuous deployment.
She shared with us how hard it is to find the right team to work with as well as the right combination of tools to reach continuous deployment.
In the end of the talk I asked her about continuous deployment in the mobile applications world and we reached the conclusion that is a wild beast due to the release process with Apple and Android.
At metropolis:lab we are already doing continuous integration until the release process with alpha and beta builds, but from that point on is very much impossible to do anything special since, when you release to the apple store, if something goes wrong, you’re in deep trouble for at least a couple of days until you can upload a new version.
She mentioned something interesting: don’t accept flaky tests.
- If tests are flaky and are still running without no one caring — delete them.
- If tests are flaky and can be automated in another layer (API, unit tests), do it
- If the flaky test is going through the same path of a non flaky one, just add the extra step to the non flaky and delete the flaky one.
I was surprised no other speakers mentioned flaky tests (or maybe they did and I don’t remember :) ) since I’ve been reading a lot about that lately…
One thing she said that makes total sense and I’ve abide by it since I started working: don’t automate just because you can.
- Automating everything leads to disaster: unmaintainable tests, flaky tests, test runs that take hours and hours to finish…
Books recommended by Amy:
- The phoenix project
- Lean startup
- Continuous Delivery by Jez Humble (I’m a big fan of Jez as well)
Dan Cuellar — Appium: An Open Source Success Story
Dan finished this conference with a funny story.
The story about how Appium came to life and how he totally lost control of the open source project he himself created :D
Dan’s story was riddled with funny moments, was a nice touch to a full day of (some) serious talks.
Again, the main reason why I go to conferences: we ended up watching a world cup match with Dan and sharing stories about travelling when going to conferences.
Dan has some big plans for Appium, he wants to make it the de facto tool to automate platform specific apps (iOS, MacOS, Android, Windows, Games, TVs, etc…).
Was it worth it?
Absolutely, yes!
I really like to go to conferences, not just because of what you learn on the talks but mostly for the networking and discussions you have with more experienced people or people that have a different point of view or way of thinking.
I really enjoyed the discussions with Angie, Bas, Amy, Ilari and Jon and how easy to access these great people are.
Thank you to Metropolis:lab for giving me the chance to attend this conference, thank you to the organisation for the excellent program and conference and thank you to all the speakers for the added knowledge to my brain :)
Hope to see you again in a few months, maybe as a speaker!
