In Defense of Interviews and Test Projects for Agile Developers: Rebuttals (part 2 of 3)

This article is part 2 of three in our Interview series. Read part 1 here »

There are definitely abuses out there, but I don’t think hiring comes down to “can you code?” That question doesn’t answer this one: are you best as a cog in a machine or a thinking partner in a business? If the latter, there’s a subjective and emotional side to the process.

And some businesses want one, some the other. You should know which you’re interviewing for and expect them to want to sort it out. Setting expectations is one of the most important aspects of running a business, and interviews are the best time to do it! Just a few expectations worth digging into:

  • Whether time spent or deliverables or both are the primary expectations of how work gets done.
  • Level of autonomy expected/desired in employees and how they can tell.
  • How competing priorities are managed.
  • Training on the job vs. training prior to the job/on own time.
  • How hard it is to work here.
  • Expectations of balancing speed with cost and quality: which are non-negotiable and which ship with “good enough”?
  • Team size and relative decision-making power per person a team.
  • Whether raises are a negotiation (rewarding extroversion?) or a standard process.
  • Work hours, dress, flexibility, and what all that really means.

We’ve chosen a pretty specific three-step hiring process at statUP, which I’ll describe in part 3 of this series on acing an interview processes like ours.

Here’s a brief review of some recent complaints. One of my favorite pieces, which is less bash-y, is this 2015 piece by Google’s People Operations leader, where he recants on silly/famous interview puzzle questions. But I want to focus on something more representative of current trends.

An HR consultant here in Austin recently shared this article in the New York Times:

The article showcases some silly interviewing assumptions and practices, then looks at some of the research on interviews. The author concludes that “interviewers typically form strong but unwarranted impressions about interviewees, often revealing more about themselves than the candidates.”

Rebuttal #1: Creative Work is Not College

I have three responses to the article above.

1. Same Facts, Different Conclusion

The part about interviewers revealing more about themselves is exactly one of the key purposes of an interview. Those interviewers will make the same mistakes as managers and teammates! Good time to figure it out.

2. Treats Jobs as Cogs

The thinking on hiring here is pretty typical, but makes extremely binary assumptions about what it’s like to work somewhere. Predictable work is handling the same customer service inquiry at Chase bank 50 times a day, or taking up the latest tech changes to a government’s internal information system — “can you do it or not” is the right question, because the “job” is predictable. In the 21st Century digital space, jobs are characterized

3. Academia is not like a Digital Job

The author’s research and examples primarily deal with college interviews and GPAs as predictors of success. What if you applied to college as a team that would go through university together? Would you merely select teammates by their high school test scores? No, you’d build a team that is different from another team that has just as much chance of success. What if you had to outwit the other teams (participate in a competitive market)? You’d spend even more time getting your team right for the style and people on it. College is not like a job at an agile business!

Interviews have a huge advantage: see how someone thinks. Because in a digital world of creative work, how you work and think is just as important as your ability to push code. Unless that’s all you want to do… graphic designers have learned to have a distaste for what they call “pixel pushing,” and the same is true for developers doing things besides maintaining ticketing systems for the U.S. Government.

Rebuttal #2: Embrace that First Impressions Matter

In communications, we should not expect the entire world to become like us just to hear what we have to say. Instead, we must think through an audience’s mindset and communicate through their perspective. Demanding that interviewers ignore their first impressions seems too demanding on the one hand, and kind of silly on the other. Train the “first impressions interviewing” habits away, yet the root causes remain outside of interviewing.

Like I said, interviews are one the best times to get to know someone. ;)

First impressions certainly make a difference, but any first impression may also be overcome. A company that values extroversion and loud, opinionated statements (fingernails on a chalkboard to me!) will still be in awe when the quiet person suddenly demonstrates some irrefutable skill. Because humans have all kinds of psychological biases, not just first impressions, you can trigger another one: rooting for underdogs is one you can go for after a poor first impression.

Does it seem unjust for first impressions to matter so much? Yes, of course. Any savvy Redditor is going to easily find ways to be indignant at almost every interview. “Can they do the job or not?”

While Reddit might want to explode at some phrase an interviewer says, training that out of the interviewer in an interview-only setting merely ensures the behavior hides itself until after you’ve signed the dotted line. I guess I’m saying that while we should always be maturing, we shouldn’t be removing our humanity from one aspect of a process and expect the rest to magically fall into place.

Rebuttal #3: Interviews aren’t “hire/no-hire” moments

They’re more than that! Expectation setting, mutual evaluation, team composition stuff, and more. Here’s one really simple illustration: the call setup for long-distance interviews.

When we interview people over the internet, there’s always the Skype/Google Hangout/whatever interoperability issues. Maybe it’s that a developer doesn’t know how to use a computer (I highly doubt it), or maybe Google Chrome suddenly is the only browser where Google Hangouts doesn’t work today… again.

Don’t you want to see how management handles this kind of thing before you’re hired, not after? I’ve seen bosses scream at employees for not having internet telephony working right, when there was no chance to get it right. Far better to see whether management is flexible enough to understand that the tools of the digital age sometimes put us off schedule. Consider the following reactions I’ve heard (from consulting and seeing inside interviews at around 25 companies)

Reaction 1: “Can you believe that? If someone was serious about the job, they would’ve set up a test call.” I, for one, wouldn’t want to work at this company and would really hope this person showed annoyance on the call. There is so much conflicting advice about getting a job, that I would never expect someone to “set up a test call” ahead of a job interview!

Reaction 2: “Hahaha you’re a developer and struggle to get basic video calling to work?” I wouldn’t want to work at a company where managers are looking to pounce on someone and make them look bad/make self look better. Hopefully this person cannot contain themselves enough during the interview and blurts something like this out on the call so you know that you don’t want to work with them — unless you do.

My Reaction Last Week: (after 10 minutes of fiddling where Skype wasn’t working, Google Hangouts didn’t, then Skype did — who knows if mic, headphone, or what issue) Candidate expresses apology and is clearly nervous, so I say “This one is on us! We’re throwing a bunch of freeware — from Slack to Skype — together to make it work. Totally not your fault. Let’s just fresh start the whole thing, ok?” If this response annoys you, then you already know we wouldn’t make great teammates :) If this response puts you at ease, then you have a little taste of what working here may be like.

Interviews help people see things that are difficult to communicate. “Do I get the job or not?” is an overly academic and simplistic way of looking at things.

What about Test Projects?

Companies are just trying to get free work from you! On Austin Digital Jobs (a great job board/community), I regularly see horror stories of abusive test projects that either take crazy-long and assume the employee wants so badly to work for the company that they’ll do anything, or actually cancels job posts after receiving free work! Ahhhh!

A recent candidate used this trend on us…

Recently at statUP we had a developer (currently out of work) refuse to do a 3–6-hour test project after dragging on for a week. He put a LOT of work into NOT doing a project and sent us a clearly advice-laden email instead. The email challenged our co-founder, Jared, on whether he had to pay anything to go to training camps to be recruited as a pro soccer player (ironically, turns out Jared had to pay a whole lot and put a bunch of skin in the game, contrary to expectation). This particular situation revealed some interesting expectations about hiring practices and how a company works.

For us, seeing how this person dealt with a simple non-statUP-related assignment is exactly why we run a test project!

Been There

As someone who poured 45+ hours into a test project (I asked for one after rejected post-interview) that landed me as first employee at a SaaS startup, I don’t have too much sympathy here.

I’ve done test projects for various hiring processes, including an extensive one for Gusto where I designed a complete feature (btw I failed this one in interview 6… but I designed something great for my portfolio! :P).

Don’t Reject for Extremes

I used to work in Government and have seen processes that would make Eric Ries weep. I remember one 19-stage, 18-month process I consulted for at a Fortune 500 company that took 18–20 months to complete, and was characterized as “everyone could say no, but nobody could say yes.” That doesn’t mean process is bad, though. It means the discipline to keep it simple isn’t easy.

Are you thinking “he’s just appealing to his own experience”? Yes, I am. I believe trying to make your company more like a computer with its decision-making is less, not more effective. The new book Hunch seems to go along with this notion. I could be super wrong here! If you don’t like this style, you probably shouldn’t want to work with me or leaders like me. That’s okay!

You Can’t Remove Decision-Making from the Process

For people looking to avoid the difficult decision-making, no method or process will do that: that is the nature of hiring. Even though double-blind interviews using endless web forms can give some appearance of objectivity, a company and its culture don’t operate that way.

Actually, not entirely true… there are companies that operate that way: Department of Homeland Security, HP, CSC, etc… if you’re looking for giant processes to attempt to bake in objectivity, there are companies that attempt to operate that way. Never mind that their executives change roles every 2–3 years, often care more about the Rolex and title than the work, and lay off 7,000 people at a time. The computer (I mean, “data”) told them to!

Test projects show you a little about how someone problem-solves and communicates. Test projects also help a candidate see if these are the kind of people that working with will be a good experience. Next week I’ll share what we’re looking for when we run a test project, with some tips for success.

Cog in Machine vs. Thinking Partner of Business

Still anti-interviews? That’s okay. I think you may be more comfortable at a large corporation where you can just pick up the next ticket, do it, and call it a day. Or maybe there’s a small business with a culture pretty different from ours that’s perfect for you!

At statUP, we want each developer to understand the business — so we do role-swapping for demo days (you might demo a marketer’s work this week), take shifts at technical support, and spend a lot of time understanding the entire goals and customers of the business.

Whether you’ll even care about that is worth exploring beforehand.

The digital world is creative space, not moving pile of dirt to from place A to place B. It’s also not an assembly line building the same Toyota Prius a thousand times. The work is largely undefined until someone defines part of it a week before, and the rest as you go. Developers can serve that process, or shape that process — and either way is an efficient method of running a company. Put another way: at our company, you’d be expected to help identify why the dirt is moving from A to B and possibly suggesting C, or in the design office of Toyota’s next vehicle.


Creative work can go at different speeds and ideas can collide based on “soft” factors such as likability, team feel, random learning, pride, shared joy, likability, and communication. If these things can make the difference, then they’re worth the time spent finding a fit.

Enough arguing!

See you next week for the final article in this series and how to ace an interview process like ours!

Like what you read? Give Isaiah McPeak a round of applause.

From a quick cheer to a standing ovation, clap to show how much you enjoyed this story.