Test automation is not a project, it’s a process

I have just read Stėphane Colson article about good practices when beginning with test automation. He says:

Automation is a real new project and should be treated as such.

I disagree. Test automation cannot be set as a brand new project, with its own team, goals and schedule. In the same way that peer review and pair testing are good practices to improve the quality of the code and the feature, automated tests are a way to make sure that, when you change something in your code, you haven’t break anything around. If tests are well designed, the feedback is fast and the team can go forward with confidence. This is only my opinion and related to my humble experience. As an example, I will begin with a short story.

In one of my first testing experience, as a QA engineer, I was asked to write unit tests for the code already written by other developers — for respect of the company, I will not tell its name. I was supposed to add coverage and to find bugs, but not to change the current code. Yes, my job was to write unit tests all day long and separated from the developers — luckily, i was allowed to ask questions about the code but not too often and only to the lead dev. After three months, the CTO saw that I was not adding enough value and we stopped the experiment — and I moved to another company. I think they continued to write code without tests after that.

Seems strange isn’t it? Everybody knows that developers should write their own unit tests, that process was a complete anti pattern. Why? Because the developer has the knowledge of the logic behind the user interface. Therefore, she knows the best what low level tests (unit, component, integration) are relevant, because those tests are there to control the logic. The issue with that process is the switching mindset, because when you implement algorithms you have to dig deep in reasoning, then go out to treat the test, and go back to modify the logic, and go out… A way to handle that can be pair programming, where one codes and the other writes the test at the same time.

And what about business oriented tests? Should developers also add the burden of that new layer, knowing that it requires a brand new mindset? I have seen that every time a different person, which is not a developer in the team, take the ownership of end-to-end tests, in parallel to development, then it becomes very hard, when the project is working (enough coverage, reliability), to say: “OK guys, now you can use and adapt those tests for your regression, enjoy!”. The answer will be, if not just ignoring it, “I have no time”, which is always a polite way to say “I don’t want to learn your stuff because I see no value in it”. Nevertheless, I really believe that functional tests are a developers tool, not a QA tool. They are there to tell dev team if their work answers to business needs.

And what if those tests are owned by test automation specialists, whatever name you give them, in a separate team? Two possibilities. Tests are run in a CI, after each push. Developers will see the errors, but they will not, at a certain point, be autonomous in interpreting the results, just because they are not familiar with the framework, with flakiness, with false positives, etc. Second possibility, functional tests can be run “sometimes”, at night for example, or even manually (automated manual tests, worse of two worlds!!), in which case the feedback will be asynchronous, with interruption, context witching, and can lead to bad relationship between teams (“I have my schedule man, I have to finish my sprint!”).

Last argument: developers are code experts and I always saw better functional tests if they are written my developers than testers who write code.

As a conclusion, I think that developers should write all automated tests, even business oriented tests, and there should not be a separated team to own those tests. Of course, for this to be useful and focused, they will need tests specialists and QA engineers in the teams, as modern testing principles recommend. And ideally, with the participation of product owners. Yes I know, this is called BDD.