Pointer is a reading club for developers. It’s a window into what other current and future CTOs are reading and thinking about.

Subscribe at www.pointer.io

Building An Effective Technical Interview Process

by Doug Petkanics

Author’s Background: Doug is a co-founder and VP Engineering at Wildcard, where he primarily builds and manages software engineering teams. Previously, he was a co-founder and VP Engineering at Hyperpublic, which was acquired by Groupon in 2012. You can follow him on twitter @petkanics.

Building a team is difficult. An entrepreneur is required not only to source talented candidates and introduce them to their company, but also to sell the candidates on their vision, their potential for execution, and on the team they’ll be spending their next career cycle with. On top of this, when recruiting for engineering positions, it’s also necessary to demonstrate the team’s technical prowess, to show interesting challenges and the direct impact that solving them will have on the company, and to prove an environment exists where engineers are empowered and integrated into the company’s strategic process.

Often times founders can accomplish some of the above in an initial introduction and conversation, but to truly sell the candidate on everything, it’s important that all of these themes permeate throughout the entire technical interview process. In order to do this, a process must be designed intentionally, such that this checklist is hit, in addition to evaluating the candidate and determining whether they’d make a great addition to the team. It’s common that the interview process focuses purely on evaluation, and neglects to deliver the candidate experience that really leaves the potential team member wanting to join on for the mission. When designing an interview process, one should not forget that these few hours that the candidate spends with your company will serve to not only recruit them to the team, but also to form the impression of the company that they’ll share with everyone they speak to going forward.

I’d like to walk through the process that we’ve put together at Wildcard, where we have a great engineering culture, and strong team with a range of backgrounds, skills, aspirations, and experiences. This process won’t be perfect for every company. We’re an engineering and product driven environment, where we value technical strength and strong computer science fundamentals; our process may look different than that of a marketing or sales driven organization where technical execution is more about implementing a spec than solving never-tried-before technical problems. Nonetheless, I believe that many of the same steps in formulating the process still apply.

Goals

In order to build the right process, it’s necessary to know what it is that needs to be accomplished. The goals of our technical recruitment process are as follows:

1. Build a great technical team.
2. Build a great engineering culture.
3. Evaluate the candidate.
4. Present the company in a positive way.
5. Create a sense of demand from the candidate.

The top two goals are more company focused, and the bottom three focus on the interview process itself. Notice that only one of the five goals (evaluate the candidate) is something that would put pressure on the potential team member, whereas the other four are really requirements of the existing team. The interview process is really more about the team, and them participating, conveying the energy and culture, and buying into the mission, than it is about the candidate themselves. If everybody who participates is bought in and excited, then evaluating, selling, and closing the candidate will come naturally. Whereas if the process is mechanical, based purely around a checklist and evaluation, then I have no doubt that the culture of the company will suffer as the team size grows. This all starts with setting the right goals and communicating them to everyone who participates in the process.

The Phases of the Process

Recruiting a new team member is not a single-step event. It happens over time, where someone who is introduced to the company tomorrow, may end up joining in two years. For an actively recruited candidate who is ready to make a move and is thinking about what’s next for their career, the phases of the process generally include:

1. Introduction
2. Screening
3. Interviews (technical + fit)
4. Evaluation
5. Closing

This post is about the technical interview process, and even though the introduction and closing of a candidate may not be purely technical, they still are important pieces of puzzle.

We find that the introduction to the company is best delivered by one of the founders. This is the opportunity to give the really high level overview, to convey the vision and potential of the company, and to make sure that the candidates motivations and interests align with the company’s needs and culture. This generally happens over a breakfast, coffee, or 30 minute long meeting at the office. It’s informal, and getting comfortable with the candidate counts for more than evaluating them. The person conducting the introduction should leave with a strong gut feeling as to whether this candidate would make a good addition to the team culturally, but should not necessitate the validation that the candidate has all the skills required to succeed on the job.

The first pass at said validation occurs during the screening. This is the first shot at a high level, broad technical interview. At Wildcard, we actually ask a technical candidate to begin with doing our programming challenge on their own schedule. The challenge consists of two technical problems that take about 30–60 minutes to get through, that they can code a solution to in any language of their choosing on their own time, and share their resulting code with us. This is a low pressure environment in which the candidate can demonstrate their problem solving ability, demonstrate they pass a reasonable technical bar, and show their interest in Wildcard by investing a little bit of time post their introduction, without all the intensity of an in-person whiteboard session. The code from the challenge serves as a reasonable starting point for the screening interview, which is usually an hour over the phone, google hangout, or in person. The screening consists of a combination of technical discussion and problem solving, and the goal is to determine whether this person has a reasonable chance of having the technical background required to get through the full technical interview process. It’s a huge time investment to invite someone to the office for full interviews, so it’s important that the screening identify people who likely won’t make it through technically.

For those who pass the screen, we invite them to a half-day at the office where they’ll meet with many folks across the company, and participate in full technical interviews. This is the most deliberate part of the process, in which we ensure that each interviewer knows their objective, what they’re evaluating for, and what they should make sure to share with the candidate. For a typical full stack engineering or data focused engineering candidate, the five different interview sessions in the process may look like this:

1. Background + Experience

What projects have they worked on in the past and what was their specific role within these projects? Go deep. Press for details. Find out if the candidate knows their stuff or if they’re just taking credit for the accomplishments of the rest of the team on the project.

2. Computer Science + Software Engineering Fundamentals

We’ll cover algorithms and data structures, networking, databases, web and/or mobile development concepts, testing practices, and more. We’ll do some whiteboard problem solving collaboratively. See if the candidate is comfortable with the value of code reviews, pair programming, and constantly learning and improving.

3. Coding

The main goal here is to determine whether the candidate is truly comfortable writing code. Do they quickly open their favorite editor, write code and tests without hesitation, look up stuff via google or docs when needed, and do they know where to go to find what they need? It’s expected that any software engineering candidate would be comfortable with this, but it’s surprising how many people expose that they just don’t have that hacker mentality at this point of the interview.

4. Architecture + Design

This is all about proficiency and instincts for larger systems. Is the candidate familiar with different patterns for communicating between different applications? Do they have strong instincts for where to split components? It’s not a requirement that every engineer joining the team has experience with this level of thinking, but certainly for senior engineers and above it’s very important.

5. Questions + Fit

Finally the candidate should have a chance to get all their questions about both the technology and the company answered, and to speak candidly about what they’re really looking for in their next role, and in their career. Hopefully they had a chance to do so during the course of the first 4 interviews, but it’s good to wrap things up with a higher level reflection of how the day has gone, and how they feel they could fit in here.

Each different interview topic mentioned above deserves its own essay to truly do justice to how to execute it correctly. The key idea is that each interviewer knows their objective, and is able to focus on a different topic, to ensure that by the end of the day the team is able to make a decision on the candidate. Oftentimes, if a team doesn’t delineate interview responsibilities, the following missteps can occur when meeting to debrief and make a decision:

* Did the candidate write enough code?

* Was the candidate challenged enough?

* Did we get through every topic on time, or did we run out before covering something that we need to evaluate?

* Did the candidate get a chance to ask enough questions such that they can make a decision?

* Does the candidate have a good feel for the company and the people, beyond just having been evaluated technically?

Hopefully the potential team member leaves the office on the interview day feeling challenged, but impressed with the company’s culture and technical depth. They should feel like this is a place where they can learn from those who they just spent time with, and they should feel inspired by the challenges that the company is taking on. It doesn’t do anyone any good if the candidate leaves feeling like the interview process was a walk in the park.

Upon completion of the interview day, the team needs to debrief on the candidate. The team has put a ton of work into an effective interview process so far, and the debrief process should be just as deliberate and effective. There are a number of models for running debriefs, but the one thing that they have in common is that the interview team needs to be able to quickly and decisively get to a clear hiring decision. In my experience, this works best when there is a facilitator for the debrief (can be the hiring manager, a bar raiser, or an HR partner) who collects feedback in advance, and runs the debrief as a well defined set of steps. For us, it starts with each interviewer sharing their instinct for where to bucket the candidate on the hire/no-hire scale. This facilitates the followup discussion in terms of who has blockers/concerns that need to be addressed, and who had a great experience with the candidate. Total consensus on hiring is not 100% necessary across the board, though it is necessary that every concern is addressed either in the debrief, or via reference checks or followup discussions. We won’t hire someone if someone has an outstanding legitimate concern. Additionally, we adhere to the guiding principal that every person who joins the team has to bring at least one elevating skill or experience to the company — that is to say, they have to add something. If the debrief is run effectively, it’s surprising that it’s almost never the case that people leave the room with split opinions. Consensus is generally formed quicker than one would think.

One thing that hasn’t gotten mentioned yet is reference checks. These are important, and ideally reference checks are conducted throughout the various steps. Depending on the candidate’s sensitivity to interviewing based on their current position, it may not be possible to do it until the end of the process close to the offer period, but they shouldn’t be ignored.

If the output of the debrief process happens to be that this candidate would make a great team member, then it’s time to move on to the closing process. A quick followup with the candidate will be most effective. Don’t worry about seeming too overeager. If all of the components of the interview procedures went according to plan, then the candidate should be excited, and the team should be excited. This is the time to get the deal done. Conveying excitement, delivering an offer, being available for questions and followup meetings to clarify things and quell any fears or concerns, all will help make the candidate comfortable accepting the offer to join the team.

The word “process” appeared in this essay 27 times to this point. That may be overkill, but I truly feel that without a well defined and executed process, a company is throwing darts and trying to get lucky when it comes to getting people to join their team. With the right process, hiring can go from really hard, to manageable and repeatable. Best of luck…and of course, over at Wildcard, we’re hiring.