Algorithmic Interviewing
I was recently interviewed for a front end web development position. In a technical interview (after the initial phone screen) I was asked to code a string parsing implementation in Google Docs while on a Google Hangout with the interviewer. No language was specified, but the rules were no use of class libraries, no internet, no use of tools, no runtime and no tests. I played along and completed the interview after about a half hour, told them thanks for taking the time to talk with me and hung up. The next day, as I was anxiously awaiting the results, and as someone who has done a number of technical interviews in the past for many different teams, I began to think about the process itself and what it said about the company for which I was interviewing.
They were employing the coding of this algorithm as a proxy for a full blown interview, either due to time and resource constraints or some other issue. In the application process I had already passed along a number of other proxies including a link to my Stack Overflow profile, a link to publicly available source on my GitHub account (including contributions to open source libraries some of which were in use by the company), links to some of my previous live projects, and professional references.
I also considered the proxies that they did not employ that I have seen or heard of other companies using. I could have shared private source from running web applications (if not prevented by an NDA), both client side and server side. For one of my projects, Ratio Clothing, we used an off the shelf cart system, but I have a full back end system written in the interviewing company’s chosen web stack that we could have reviewed and discussed. I could have also coded an actual small scale web application, like a micro blog engine. I have been asked to do similar things in job interviews before. Every single one of these proxies that they did not employ was better aligned with my potential day-to-day work at the company and the kind of business value I would produce.
So why didn't they choose any of those?
My guess is that their answer to that question would be time. You would need to have an employee review the person’s code or open source contributions, or review their coded blog engine, and maybe they think this takes less time. And maybe this company was just employing this technique as the first line in their technical interview process. After this step, you would go on to possibly do one of the other steps I described above or even just a free form conversation about web development. So in this step they are just trying to weed out developers who are wondering how to open the database in HTML or think web development is just Googling for the closest matching jQuery plugin and find those they are actually real web development candidates. So if that's the case, why not ask a web development question?
But even if you ignore those explanations and alternatives, if you look at what they are trying to accomplish, are they even doing that? Are they optimizing for the right developer? Stripe recently posted on Quora about their interview process:
Our interviews try to simulate the work you'd do on a day-to-day basis. We don't ask any purely algorithmic questions — no project at Stripe has ever required writing a red-black tree from scratch. We're also perfectly okay with people Googling around or collaborating with their interviewers.
This clearly points out the fundamental flaw in algorithmic interviewing. It does not match day-to-day work and it is optimizing your interview for the wrong developer. Most companies are looking for good, efficient, business developers. They do not want developers who will reinvent the wheel on well worn territory unless there is actual business value to be gained there.
There are lots of problems with reinventing, especially in areas like cryptography (see another recent article from Hacker News, You are Dangerously Bad at Cryptography). Business value is the other problem. If you look at a company like Stripe, there is probably a lot less money to be made from writing a red-black tree from scratch than there is in improving their API for developers or writing language specific libraries.
In terms of company priorities and ROI, algorithm reimplementation is very, very low on the list.
Being a great business web developer and being a great algorithm developer are not mutually exclusive, but they are also not interdependent. The type of developer that would perform well at this kind of challenge is potentially a recent graduate, fresh from an algorithms class or one that is more focused on micro-optimizations than the big picture. Of course with this process you may find a developer great at both algorithms and the web, but you have optimized your process for the passing on the wrong type, and as such, your future steps in the process have a higher probability of wasting time, potentially even wasting more time than just interviewing for what you want from the very beginning.
My experience gave me the feeling that this was more of an interview assembly line than someone really interested in my relevant skills, using a kind of shorthand for the technical interview process designed to minimize time spent, ultimately not the type of company for which I would want to work. The experience is in stark contrast to my experience with my current employer, MojoTech. Not only was the interview more in depth (after the initial email and phone screen) it left me feeling better about the company, as opposed to worse.
The bottom line is there is no shortcut in hiring good people, you need to spend the time and put them in a situation as close to possible to what they will be doing, whether that's a probationary period, short term contract, or just having them do some work over a weekend, it will ultimately give you much better hires in the long run.
Some other people on this same subject:
How do you feel about brainteaser type questions:
We used to do these in our interviews. When I interviewed at thoughtbot I was asked about Fibonacci and a few Big O type questions. We’ve sinced stopped that. I don’t think it tells you much about someones ability to do what we do for a number of reasons. We don’t typically write algorithms. The correct answer for how you do Fibonacci is look on Stack Overflow. If somebody is doing anything but looking up the answers to pre-solved problems they are not being a very good consultant.
Joe Ferris, Thoughtbot CTO, from Episode #59 of the thoughtbot podcast
We found that brainteasers are a complete waste of time. […] They don’t predict anything. They serve primarily to make the interviewer feel smart. […] what works well are structured behavioral interviews […] where you’re not giving someone a hypothetical, but you’re starting with a question like, “Give me an example of a time when you solved an analytically difficult problem.”
Laszlo Bock, senior vice president of people operations at Google,told the New York Times
