Testing Testers: How to hire quality

Recruiting for a new hire, or finding a new job of your own, has never been easy. It becomes even more difficult when your profession is poorly known and often ill considered.

If you read blogs or magazines dedicated to software testing then you have probably encountered some of the common misconceptions about testing: that testing is mostly a repetitive and boring task, that testers are solely responsible for errors on production servers, that everything can be automated and therefore the job of the tester will disappear, that testers are unskilled developers, etc.

If the recruitment process is managed by people who do not understand testing, then they are unlikely to know how to evaluate testers. If they cannot see how a person could be a good fit, they might hire someone not suitable or miss that special someone.

As a candidate in front of people whose recruitment methods are the same for a tester as for any other role, then you have little chance to really understand how great the job could be. You may run away thinking that the company has a bad culture, or that testing is not valued by engineering teams, or that you will be the lone tester considered as a bottleneck.

We will see in this article how to define what might be the characteristics of the candidate you are looking for, then how to attract good applicants, and finally how to evaluate them with an interview or an audition.

What are the characteristics of a good candidate?

Obviously, there is no precise answer. It will greatly depend on your actual need, the team that the applicant will have to join, and what you imagine as the future of your environment.

If you’re looking for an automation engineer, then of course you’ll need to find someone that knows how to code. But if your framework uses Ruby, is it a good idea to dismiss the expert python candidate?

A good tester must have the ability to learn quickly. Whatever position you offer, it is almost certain that what you think your need is today will become obsolete within a couple of years, and this is not only true for automation but for all testing experts. A good candidate is one that will stay with your organisation as it evolves

Even if almost everyone agrees that the requirement is useless, there is still a tendency to read “ISTQB required” on some job offers. I guess it’s because it’s very easy to sort people that are certified and people that are not, and then quickly select a few to enter the recruitment process. Do you really think that someone who has passed an exam is definitely a good candidate?

Certifications are useful if you can have the list of people who failed the exam, because they were not able to learn or study enough to be sure of their success. Sadly you can only know who succeeds, and success proves only one skill: the ability to pass an exam, which is not the job you want them to do. Certification isn’t an indicator of good, and you will meet a lot of good candidates who have no certification and bad candidates who are certified.

A good tester must be curious. Firstly because they cannot spend all of their testing career doing the same tasks. They need to constantly improve, learn new things, and without being very curious about everything around their craft they cannot be motivated in learning new stuff. Secondly because they must discover a wide variety of information about the business and the technology involved in the product they’re working on.

They also need to be a good communicator. A tester is the one who will have to communicate bad news — “Houston, there’s a huge problem here”. They are also the one who must communicate with people that don’t like to communicate, which includes some developers. They talk to management, product owners, and sometimes with customers, not only in face-to-face conversations but also using many tools: email, instant messaging, phone, bug tracker tools, etc. A good tester needs to communicate in a diplomatic way, without being too intrusive and without being ignored. That’s a difficult skill.

How to attract a good applicant, and how to retain them?

Certainly not in lying. Whether with an offer, an interview, or an audition, you will have to make them feel you have the best job to offer, but do not say things that you won’t be able to offer.

If you can offer flexibility in schedules and holidays, these are things that might be appreciated, maybe more than a good salary. Good testers place a great deal of importance on progressing in their field of activity. If you can help them to attend the conferences of their choice, do your best to make this happen, and don’t hesitate to sponsor those conferences: by doing this you will show how you value learning in your teams.

A good company culture is one that enables everyone to give their best. That’s why a good candidate will prefer a company that values mentorship over old-fashioned management without trust or flexibility for experimentation. Can you offer this and prove it?

In a previous edition of Testing Trapeze, Daniel Barrow shared how he was convinced that PushPay is a great place to work. I loved the idea of first having a coffee while talking in a relaxed and informal way about the company culture, and not only about your CV.

Resumes and cover letters are still requested, sometimes using the company template. Some companies also send a form containing many questions that are not related to the job that will have to be done. I’m sure that these practices are very damaging to the image of the company and are deterring a lot of candidates. Maybe the best one is among them and you’re about to lose them by asking for this.

How to efficiently evaluate an applicant?

The difficulty when you have candidates in front of you for a short period of time is to evaluate them correctly. To imagine them integrated in your team, while guarding against the cognitive biases that you might be the victim of.

Beware of confirmation bias. You could make the mistake of deciding the fate of the candidate in a few short seconds. You might quickly have a negative image of an introverted person, and we know that introverts can be excellent testers (poor sales people, but that’s not what we’re looking for, are we?!) . Or you might overvalue someone that knows how to introduce himself efficiently but is a poor candidate compared to other qualities expected.

Watch for Status Quo bias. When choosing between an average candidate with knowledge in a particular programming language and another candidate who does not know this language at all but may be able to learn in a few weeks, it might be better to prefer the latter.

It is now popular in IT to prefer auditions over interviews. This is a very efficient way to test an applicant. You can ask them to perform some testing on your application at home, or you can ask them to come into your organisation to work a few hours or days directly with the team. The candidate might do some pair testing, add some automatic checks, or fix a failing test if that’s part of the job you propose.

You and your team will gain a better understanding about the candidate when you will see them work in real conditions, directly using the tools you will ask them to use. The candidate can share work-related knowledge with the team members you will ask them to work with. If the audition is successful, then that’s a good way to test the fit between a potential employee and a company, on both sides.

You must be aware that not everyone will have time to engage in this process. Some have a job during the work day, a family to spend time with in the evening, and commitments during the weekend. Some candidates will have several opportunities to evaluate and won’t be willing to block one or several days of their calendar just for you, unless they really want to work for you, and only for you. Additionally you should pay the candidate for the time they spend working for you. You may not be able to do this with a lot of candidates.

If you cannot do auditions then you can evaluate the candidates by asking them to think about some quick testing tasks. For example:

• ask to design a test. Prepare a user story (add bugs or incomplete information inside), ask them to write acceptance criteria and see what happens
• ask them to talk about how they would test something easy to understand, a very small piece of software or anything else (a bulb, a coffee machine, the trackpad of your laptop…)
• you can use puzzles like these ones and evaluate his ability to explore
• prepare a dev environment and ask them to add a check in your automatic test framework if it will part of the job
• or ask them how they would interview and evaluate a candidate for a testing job

Conclusion

As a company you are not alone in the market. You must have the best showcase to attract good candidates. Your hiring process should reflect how awesome the company is and evaluate the candidate according to the job they will have to do.

Testing skills are very specific and not easy to assess, a lots of biases are to be avoided on both sides. You can audition the applicant and make them work with the team or use some simple exercises to see how they manage them.

As an applicant, finding a good company where you will be allowed to do good testing is a challenge. Some will have very poor hiring processes for awesome jobs. Others will have very shiny and sexy hiring processes for boring jobs.

Hiring needs to be improved in order to achieve its goal to find the best fit for the best candidate for the best job. That’s not easy in software testing.

This article was originally published in Testing Trapeze in June 2017. Testing Trapeze sadly stopped being published and is no more accessible online.

Originally published at lyontesting.fr.