A bit about me: I joined Google right out of school in Singapore, and my first full time project belonged to Web Search Infrastructure group. That was 2007. By last year, 2013, I’d gone through 5 different projects all belonging to Web Search/ Knowledge infrastructure within Google. I’m a big fan of massively parallel, distributed, low latency systems. They impose very interesting challenges related to threading, network communication, asynchronous / synchronous callbacks, and of course logic. I strongly believe that if a system is to be performant, you can’t add it on top as a functionality. You have to design the system from scratch for performance, weighing out every extra bit of complexity against functionality gains.
The interviewer: While at Google, I interviewed around 50 candidates, and my favorite question was to dig into open ended design.
Over time, I established a really nice rule to figure out if the candidate did well. The rule is simple:
A: Did I have a great debate about the pros and cons of the approach taken by the candidate? Did we agree a lot? Did we disagree a lot? Did I learn something new? Overall, did I have a great conversation, the likes of which I’d want to have with him / her as my colleague?
B: Did I have to spoon feed thoughts to the candidate? Was he really slow? Was it an effort to keep the conversation alive?
Case A was amazing. I loved those interviews. I felt energized when a candidate challenged me in my thinking, could think of alternative solutions, and why they would or wouldn’t work. The best of my candidates were the ones who designed something, admitted in the middle that what they designed was flawed, and they’d do it from scratch with a new methodology. I was particularly impressed by them, because their love for perfection was above their ego.
Case B were no hires. It was an effort even to write feedback for such candidates.
And then there was a whole spectrum in between.
I think a great engineer resembles a painter. He’d design something, learn what works, what doesn’t. Throw it away, and design better. Most importantly, he’d be hard to please with his own design.
The interviewee: It had been 6 years at Google, so I decided to work for a smaller company. I left Google, and started interviewing. And I was surprised!
Interviewers were asking me design questions, and I was trying to have a great conversation, a debate, thinking / rethinking pros and cons of various alternatives. But, a lot of them weren’t looking for that.
What were they looking for? Think of an algorithm question. The question is defined. The answers are well known to interviews. There’s the typical brute force solution, then there’re improvements. Then there’s the best answer, which has best space-time complexity possible. If the interviewee can come up with the best and code it, you got a happy interviewer. This is the standard.
What was surprising was, these interviewers who were asking “open ended design questions” were stuck with the same mindset. They were looking for a particular flow. They were looking for a particular answer, or a particular question that the interviewee was supposed to ask. What astounded me was how little they cared about entering a conversation, a discussion, a debate about alternatives.
Recently, I interviewed at a really well known company in Berlin. I was asked to design a key-value store. I started with explaining a design similar to Memcached. Then went onto explain the design for a persistent store, like Bigtable. Then went through the failure scenarios — if master crashes, if worker crashes, if the network goes down, if the datacenter goes down etc. I talked about replication, talked about sharding, talked about backups. I think within that hour, I’d discussed elaborately about the pros and cons of every piece and aspect of design I could think about.
The interviewer, apparently, wasn’t satisfied. He complained that I didn’t ask the “right questions.” I was rejected. I remember how we started the conversation:
Interviewer: Design a distributed key value store.
Me: Like memcache?
I: Memcache isn’t distributed.
Me: Well, it actually is. The Memcache client supports sharding by default. And uses a continuum based consistent hasing approach, which allows us to add or remove servers without affecting all the keys… You only affect 1/(N+1) keys, yada yada.
I: … nothing ..
Me: Hmm.. alright, so here’s how a design like Memcache would work..
And so on..
I can sense when the interview doesn’t go well. Even though, this particular one went really good in my opinion; I could feel that the interviewer wasn’t into it by the end of it.
How do you know if the interview went well or not? You get 5 mins at the end of interview for questions. As an interviewee, you can gauge by asking the interviewer a question about the interview process (don’t ask how the interview went). If the interviewer didn’t feel good, he’d shy away or be awkward about it.
I sensed it. After the interview, I asked myself honestly. Would it be possible that the interviewer rejects me? And my experience told me, that’d be stupid. I explained so many aspects of the system, it would be hard to ignore all that “meat”. And it very well, turned out to be very idiotic.
I think that’s deeply flawed.
And the reason this happens is lack of experience of the interviewer, and hence lack of sensible reasonable expectations for the outcome of the interview. If you’re an interviewer, asking design questions, please do everybody a favor: Enter the discussion. See if you can have a great conversation. Keep yourself open to ideas, new ways of thinking, even though they might conflict your established design. Ask the candidate lots of questions, challenge him. You might just find yourself a colleague you’d want to be around.
In any case, don’t sit quietly looking for a “particular question”. If you’re one of those guys, stick to algorithm questions, don’t venture into open ended design territory. You suck at it!
About the cover: Van Gogh wasn’t impressed with his now most well-known and admired work.
In a letter to his brother, he describes the painting:
The first four canvases are studies without the effect of a whole that the others have . . . The olives with white clouds and background of mountains, also the moonrise and the night effect, these are exaggerations from the point of view of arrangement, their lines are warped as that of old wood.
Leonardo Da Vinci held onto his painting of “Mona Lisa”, without releasing it, continuing to work on it for years, stroking a brush now and then. It was drawn on a normal size canvas, at somebody’s request. He didn’t think of it as his masterpiece. It was supposed to be a painting for a gallery, to be hung along with many others. Definitely not deserving of an entire wall, in his opinion.
Engineers can learn a lot from such remarkable painters.