Hiring a tester — Second part: Evaluation

We’ve seen in a previous article how to evaluate a tester with a resume, a motivation letter or any other media (Linkedin, job board), etc.

Now that we sent back home all candidates who doesn’t match, let’s go ahead with a first interview.

Phone call or face to face?

The answer to this first question depends on several parameters. Is it a better idea to have a direct meeting, a phone call, a video conference talk or an instant messaging conversation?

If the candidate is not in your area and if you are looking for someone to work remotely, then you’d better not ask him to come in your office first. Even if he (or she) is in the same area, then you might also prefer a remote contact for the following reasons:

• an introvert, someone very shy or easily impressionable or stressed will be more comfortable than visiting your unknown office. During an interview, there is a big unsteadiness between the interviewee and the interviewer(s). The interviewee does not know very well the itinerary and the place where you will interview him. Maybe that he does not know anyone in the enterprise, and he does not know the dress code and can feel weird if he discovers that he did not chose the adequate clothes (“oh god, why did I chose the tie? everyone is wearing a t-shirt”). If you invite several team members to attend the interview (everyone knows each other, the interviewee seems lonely), then the candidate can quickly panic and not act at his best. Competing with someone comfortable in all circumstances, it is highly probable that you will chose the latter, even if nothing proves you that he will be the best team player and the best tester.
• if you miss a point and quickly found that the candidate does not fit at all or seems not enthusiastic enough, then it will be easier to shorten the conversation and avoid losing too much time
• you will be less disturbed by cognitive biases because of the look of the person, or because of non-verbal communication. On the other hand, if you preferred a phone call or a writing contact, then you’ll have to be aware of other biases, for example thinking that someone speaking perfectly your language will be a worst team player than someone with high communication skills.

"Most interviews are basically an exercise in confirmation bias." - [Simon Knight](http://sjpknight.com/)

Meeting people in real remains very important, and if you have to, you should consider doing it in 2 steps in order to avoid the risk to spend a lot of time for nothing and also for a better evaluation. I’m personally in favor of a first phone call with the objective to simply confirm or not the first impression done by the resume and the motivation letter (if any).

We can imagine, in the case of a tester interviewed, the following steps:

• Each one introduces himself briefly
• You explain the context of the project and the enterprise
• Candidate explains his experience and his personal assets
• Candidate explains what is his major experience (not the latest one if it’s not an interesting one)
• Ask the candidate an easy question about the tester role, for example:
• What is the difference between Black Box testing and White or Grey Box testing?
• What is exploratory testing?
• Ask the candidate (if he was correct with the first simple question) a much harder question, for example:
• How to chose what must be automatized and what must not be (I know that there is probably not an easy good answer, but the goal here is to start a conversation, and see if the candidate ask the correct questions or if he has preformatted ideas due to too much faith in some certification, not willing to question things more deeply)?
• Ask him how he would test a feature: for example, how to test a localization service which fills automatically an address during an account creation?
• What kind of tests are specifically important for a web application or for a mobile application?
• How do we know that we must stop testing?

Finally ask the candidate if he has some additional questions

Starting a conversation using an instant messaging app can be interesting too if you will have to use it during your future eventual collaboration. Having a contact with the candidate with Slack (or any other alternative) to ask him some additional information, or just to keep contact and see how he behaves with writing communication (and not with a motivation letter he gave to all his family for a review, also helped by auto-correction tools) can help detect potential issues like:

• He is not a good communicator using this media, he is messy and you cannot understand him
• He writes like text-messages to friends, using lots of smileys (omg!), and you know that everyone in the team hates this
• If you need him to communicate in a foreign language, it’s a good way to ask him to write in this language and evaluate his writing skills.

If this first contact is conclusive, then both of you didn’t lose too much time and you can go forward.

Don’t make him run away

During one of the first contact with some companies, I’ve been asked several times to rework my resume in order to apply it with the graphic chart of the company. Hearing this, I first thought that the goal was to only add my information in a recruitment database.

This is awful, specifically when a non standard .docx document is attached, because when I opened it with another tool that the one used to create it, then it was displayed badly.

The candidate must be very motivated to spend hours fixing this and adapt the template with his own resume information. Remember that the candidate already spend lots of time on his own CV, and also on social media like Linkedin.

If you are a recruiter, stop asking this; trust me, there is no good reason for asking this. And if you think you (or your client) need it, do it by yourself (preferably with Open Office or Pages), and you will understand how boring it is.

Evaluation

Tester role implies a lot of contacts with different kinds of other roles (customer support, developers, product owners, etc,…) which means to have qualities of a good communicator (writing and oral). Neglecting this point in order to only evaluate technical skills would emphasize a team member who won’t be listened, not promoted and not trusted: in 2 words “not useful”.

I advice these 2 actions now:

1. Give to the candidate a small task to do at home, at his own pace. For example, you can ask him to test and find issues in a mobile application or a web site, then to write a bug report for these issues. If your product is lucky enough to be open sourced and easy to test (by this, I mean that it does not need to recompile the Linux kernel with 43 steps of configuration) and immediately usable, then it’s better to ask him to test your product than any other one.
2. Ask him to come in site to work with members of the team. Why not ask him to do some pair testing, or just some exploratory testing and see what is raised? If the role is for automation of some checks, you can ask him to add a new one. This candidate, if he joins your team, will spend a lot of time with it, then it’s very important to gather several thoughts from several members and see how he behaves when temporarily integrated in the team, and also how the existing members works with him. Here, it’s not about blocking his calendar for several days (unless he’s ok and paid for that), just a few hours can be enough to get a good feedback.

As a mini-task to evaluate the candidate, here’s a non-exhaustive list of other exercises ideas:

• Test this small form which checks if a text is a palindrome or an anagram: http://xndev.com/palindrome/
• In the same philosophy, you can find some “puzzles” on the Internet like those created by James Lyndsay: http://blackboxpuzzles.workroomprds.com/
• Ask him to write one or more test cases for a simple example to test (date or email field). By being vague, you can evaluate if the candidate has the right behavior by asking questions in order to apprehend the context and scope of what is to be tested
• Ask him to design tests and write some acceptance criteria (with Gherkin syntax) by giving him a User Story
• Ask him to perform some exploratory testing

Should you ask him to come to your office and also ask him to work on a task at home? It depends on how you trust your feelings and first impressions. But it also depends on his available time, you should not forget that he may have other interviews in progress with enterprises ready to propose him a collaboration right now, waiting for the trial period to be sure of their choice.

Conclusion

At the end, a last interview to discuss and see if the candidate is interested by the role, if you are interested by this candidate and if everyone is on the same track is a must-do. But it has nothing specific to the software testing role, that’s why I won’t write about this.

In a nutshell, no-one knows better how to hire a future additional team member. You must, just like in testing, experiment, fail, pivot, be driven by the context, learn, improve yourself and always constantly challenge the process and yourself. Don’t try to be perfect on the first try and don’t believe in best practice. An idea that works fine in a context can be a mess in another one.

During my research, I found that I was not the first to write about this subject, and found lots of awesome articles about hiring. Let me share them with you:

As usual, comments are highly appreciated.

Originally published at www.lyontesting.fr.