Hiring Software Engineers

Over the past 15 years, as an engineering manager, startup founder, and startup advisor, I’ve hired more than 100 engineers. Here are some notes on what I’ve learned …

If you’ve spent much time around startups, you know that hiring, in general, is critical, time-consuming, and challenging. That goes double for hiring software engineers, because there is always a shortage of qualified programmers.

Even so, it’s very important to hire carefully and well. Which means only bringing into your company people who will be good colleagues. A bad hire in a startup creates big, big problems. Unwinding a bad hire takes enormous amounts of energy, costs money, soaks up a lot of time, and affects everyone’s morale to a surprisingly large degree.

So, how do you hire well? The primary goals to keep in mind are: you want to learn as much as possible during the interview process about what it will actually be like to work with someone; and you want to do everything you can to avoid mistakenly hiring someone who will be a bad colleague.

Over the course of fifteen years as a founder, friend, and giver of advice to startups, I’ve worked out a process that I try to follow.

1. Write a good job description.
2. Throw out all the resumés that didn’t follow the directions you provided about how to apply.
3. Respond to applicants with a quick no, or a detailed yes.
4. Do an initial phone (these days, video) screen.
5. Give the applicant a non-trivial programming exercise to do at home.
6. Bring the person into your company for an all-day interview that includes more programming, plus some unstructured social interactions.
7. Check references.

A Good Job Description

You need to write a good job description for three reasons.

First, the job description is part of “recruiting,” meaning it’s an element of how you attract great candidates. What’s unique about your company? What’s exciting about the job? Programmers tend to like specificity, so give details about what the person you hire will be working on and the languages, tools, and platforms you use.

(Recruiting — finding, sourcing — candidates is a full topic of its own, and worth a whole ’nother post).

Second, a good job description appeals to a broad pool of candidates. Read this piece by Lisa Tjapkes on writing gender-neutral job descriptions, and these tips from Model View Culture on building diverse teams.

Third, you should provide specific directions about what you want applicants to send you, so you can …

Throw Out Resumés From People Who Didn’t Follow Directions

Here’s one way to invite people to apply to a job:

To apply, please send us email (jobs@pluot.co) with a short note about you and why you’re interested in this job, a resumé in plain text or as a pdf, and (if you can) a bundle of or pointer to some code or tutorial text you’ve written.

Having provided specific directions about how to apply, throw out all the applications that didn’t follow your directions.

When I first started hiring people via job postings — hiring people I didn’t already know — I was obsessed with not missing out on any potentially great colleagues. Good programmers often march to their own beat, I reasoned. Maybe somebody who doesn’t include a cover note, or sends a resumé in .doc format, would do great in this job.

I was wrong. I have interviewed well over a hundred people for programming jobs. People who don’t follow application directions never do well in interviews.

Initial response

I’ve always tried to respond to every application I’ve received. For the people who aren’t a fit, just a couple of sentences thanking them for the application and saying that you aren’t going to interview them is enough.

For the people you do want to interview, it’s worth setting expectations right up front about your hiring process.

Hi Jill,

Thank you for your interest in what we’re doing at Pluot and your thoughtful cover letter. I’d like to talk to you more about our work and how you could contribute. We have openings for development positions with a heavy algorithmic bent, and also for application-level programming.

My guess is that you might be a very good fit for the work our machine learning team does, programming which encompasses algorithms development, writing scalable code that operates on large data sets, tooling and ui development for our internal use, and lots of optimization/testing/tinkering! Most of the work is in c++ (though, as you might imagine, we write ancillary code in matlab, r,
ruby, java, and other environments).

If that sounds good to you, our interview process would look something like this:

1) video conference conversation
2) an at-home programming exercise which would probably take you a few evenings of work
3) an all-day interview at Pluot including conversations with four or five of us and another big chunk of programming

We know this is a fairly intensive interview process. If you’re willing to spend time with us, though, we’d really appreciate it. We think it’s worth going through these steps to make sure you’re a great fit for this job and we’re a great fit for you. We promise to be respectful of your time. Would you like to schedule the video conversation?

Yours,
Kwin

A small percentage of people who seem very qualified on paper will object to doing a significant amount of programming during the interview process. That’s okay — they just made it very easy for you to decide not to hire them. It will be just as little fun working with those people as it is “negotiating” with them about how they want to be interviewed.

The Phone Screen: Low Expectations

The phone screen is the traditional first step in getting to know an applicant. I have always found it hard to get much useful data from a phone screen. So, these days, I have relatively low expectations for learning much about someone during an initial phone screen.

In particular, I don’t try to do any programming exercises, or logic puzzles, or anything like that.

Instead, I try to accomplish two things. First, I’m 80% in recruiting mode, during the initial call. Experienced programmers will always have multiple job opportunities. I want the applicant to be excited about the one we’re offering her.

Second, I try to get a rudimentary sense of what the person will be like as a colleague. To that end, I do all initial conversations as a video call, and do them together with another person on my team. (At our company, we make affordable, super simple to use, video conferencing hardware, so it’s easy for us to do this!)

Having two people from our side in the conversation, rather than just one person, tends to give us a better sense of how an applicant communicates.

Programming Exercise

I’m a big believer in asking people to write something non-trivial as part of the hiring process. In fact, it’s one of the two things related to hiring engineers that I feel most strongly about. (The other is that it’s important — for a lot of reasons — to make an effort to hire people from a diversity of backgrounds.)

If the job you are hiring for involves writing code, the best way to get a sense of whether someone will be able to do that is to ask them to write code. Of equal importance: writing code is a relatively solitary discipline, but planning, reasoning about, fixing, and maintaining code is very much a collaborative activity. So asking someone to write code and then show it to you and talk you through it, is an extremely useful simulation of what it will be like to work together.

Here’s a programming exercise that I’ve used many times to hire javascript programmers.

We’d like you to build a single-page javascript application that lets users drag colored rectangles around an area. The application should be as simple as possible but also needs to work robustly and intuitively. It’s fine to include documentation on use, but if the application itself gives the user enough feedback to figure out all the possible activities organically, that’s even better.

Here’s the requested feature set:

— add a new “rectangle” to the area
— remove a rectangle
— resize a rectangle using direct mouse manipulations
— move a rectangle using direct mouse manipulations
— change the color of a rectangle
— clear the area
— save the current area layout (the position, size, and color of each rectangle in the area)
— name a saved layout
— return to a saved layout
— delete a saved layout from the set of saved layouts

Please use any toolkit(s) that you like in implementing this application, then put the application up on a server so that we can play with it “live”, and provide a link to download all of the code that you’ve written. Please do document which browsers you have used to test your code.

Don’t hesitate to ask any questions that you may have. We love questions (and we know that the above instructions may not be completely clear).

This is a very open-ended exercise. I want to see how the applicant approaches a task that is incompletely specified. Real-world programming tasks are always incompletely specified!

I’m interested in tooling and architecture choices and in how much the person focuses on the “back end” versus the user interface. I want to get a sense of programming style: comments, code organization, consistency of formatting and naming conventions, handling of language quirks.

Finally, this is a test of whether the applicant can follow through on a task, and (again) is willing to follow directions.

If an applicant submits code I think is of poor quality, or doesn’t follow the minimal, but clear, directions in the second to last paragraph, then I don’t make the hire. My bar is fairly high for this exercise, though I do discount significantly for inexperience if the role being filled is entry level. I want to see an applicant make an effort, and write code that’s generally good. That’s the whole point of this step in the screening process.

The All-Day Interview

For applicants who do a good job on the at-home programming exercise, the next step is an all-day interview.

If an applicant isn’t local, fly them in and offer to put them up overnight in a hotel. Paying for travel is a lot cheaper than paying a recruiter to find you candidates. And, while I’m the biggest believer in the value of video conferencing that you’re ever likely to find, there’s no substitute for an all-day, in person, interaction with everyone on your team. (That’s true even for candidates that you are hiring to work remotely.)

The all-day interview is your opportunity to learn as much as possible about what it will be like to work with the person you are hosting. They should spend time with the people they will be working directly with. You (collectively) should be partly in recruiting mode, as well as evaluation mode. (Again, if the candidate is good, and experienced, she will have many job options.)

I think it’s useful to have the applicant do another chunk of programming during the all-day interview. If you have internal platform code that you use, you can point him at your internal documentation and ask him to write something that makes use of your platform. And make it clear that asking questions is not only okay, it’s encouraged. You learn a lot from what kinds of questions someone asks, and from how they absorb your answers.

Another option is to have the applicant extend the code from the at-home exercise in some way. Whatever task you ask him to do, have at least two people spend a significant amount of time with him, talking through the code he’s written. Again, writing code is part of being a working programmer; communicating about code is another big part.

Finally, it’s great for a group of people to take the applicant out of your office for lunch, or dinner, or both. Unstructured social interactions with your team are useful, and might surprise you, in both good and bad ways.

After the applicant leaves, it’s time to decide whether to hire her, or not. Different companies make these decisions different ways. But, however you make the decision, make sure that you solicit input from everyone who spent time with the applicant. In particular, if the applicant made anyone on your team uncomfortable, that’s worth paying very close attention to. It’s distressingly common, for example, for male programmers to be dismissive of or disrespectful to female engineers. Sometimes they don’t even know they’re doing it — it’s just habit. But, habit or not, that behavior is not okay, and it’s toxic, and your “no hire” decision is made for you.

Checking References

The final step before making an offer is to talk to people who the applicant lists as references. I always ask people to give me three references: someone they have reported to, someone they have worked with as a colleague, and someone who has worked for them (if they have ever managed people). Then, in addition, I find my own back-channel references that the applicant didn’t list.

I used to be a little bit cavalier about reference checks. I figured that if I spent a lot of time interviewing people, I didn’t necessarily need to talk to people they had worked with before. Plus, I wasn’t very good at doing reference checks, so I didn’t understand how useful those conversations can be.

Then I got sued by someone I had hired on a recommendation from a friend of a friend.

It’s a long story, but the short version is that the person who sued me (and my colleagues, and our company) was unproductive and dishonest, faked an injury, exploited the worker’s comp system in order to make it difficult for us to fire him, and sued us when we finally did. I hadn’t done any reference checks before hiring that gem of an individual. But I believe this was not the first time he had done something similar, and I will always wonder if I could have seen a red flag if I’d taken the time to do reference checks.

That experience made me get serious about doing reference check conversations, and doing them well.

My main goal when talking to references is to listen for dog-whistle indications that a person may be difficult to work with in ways that I haven’t already picked up on. In the first part of a reference check conversation, I make it clear — truthfully — that we are inclined to hire the applicant and that this is the last step before making her an offer.

Usually, people say lots of nice things about the applicant, in response. I get a little worried, though, if I don’t hear very specific and positive praise, because most people want to be nice, helpful, and supportive of their former colleague. If they’re not really supportive, they may not have much that’s truly positive to say.

After the first part of the conversation runs it’s course, I try to ask a question that will be genuinely helpful in managing the applicant, and also can give people room to indicate any criticisms that they have, but might otherwise be unwilling to voice. Something like:

Based on your experience working with Jill, could you tell me what I’ll need to do, as a manager, to make sure that she is as productive and happy as possible, at Pluot?

Finally, it’s worth noting that reference checks can actually be fun conversations, and can make you even more excited about hiring someone. My favorite way this happens is when the person you are calling as a reference, says, more or less, “Hey, I looked at what your company is doing when we exchanged email to set up this call, and it seems interesting, but what I really want to tell you is that if you hire Jill, I hope there will be openings on her team, because I’d work for her anywhere.”

Startups are crazy roller coasters. Riding the ride with a great team is both one of the best things about the experience, and critical to success. Do yourself — and everybody you work with — a favor. When you hire, devote all the time, energy, and brain cells you can to bringing the best possible people on board.

My current startup is Daily.co — we make super-simple video conferencing software and hardware. We’re obsessed with helping distributed teams work well together. We’d love to hear your feedback on what we’re doing.