How Teachable Revamped the Backend Take-Home Assignment
I joined the Teachable team two months ago (mid-September). I felt throughout the interview process that it was friendly, candid, and conversational. Most importantly, it was evident that the engineering team was looking for quality coders as much as they were looking for quality teammates. I could tell Teachable wanted engineers who believed strongly in the product’s future and wanted to build it right.
Like in all engineering interview processes, there was a technical component, and Teachable’s was a take-home test. The assignment? Write a Ruby gem that wraps a basic mock API. The directions were sparse:
Welcome to the Teachable Mock API
Below you’ll find a description of our available API endpoints
[example curl commands for each endpoint]
That was it. The API was pretty straightforward: register a user and then get that user’s data, then read/create/destroy a user’s orders. Aside from being told to write a gem to wrap the API, I was given no other constraints. Slightly excited but nervous about the apparent simplicity of the assignment, I dove into writing my (first) gem.
I was initially anxious. Given this would be the first gem I’d author, I wanted to be certain I went about setting it up it correctly. I remember wondering to myself, “Were the ways in which I configured the gem all anti-pattern? Were they simply wrong?” And, “How should I architect everything?”
Fortunately, I began to relax pretty quickly, because I realized that I could build the gem how I wanted it thanks to the open-ended directions. It could be as simple or complicated as I wanted it to be. I opted for simple since nice, legible code is also highly maintainable. I reasoned, who wants to work with someone whose code is impossibly complicated just for the sake of being complicated? In addition, I made sure to build lots of tests and thought carefully about what specifically needed testing.
As an example, I weighed heavily whether to use cassettes versus simply stubbing expected responses as I wrote my specs. I decided to use cassettes. This may seem inconsequential, but we did discuss this decision in my in-person interview, and it was beneficial to easily share my thought process. The point is: by being intentionally aware of my reasoning as I built the gem, I was able to back up my thinking in-person.
Ultimately, I wrote a gem with some minimal code that hit the described endpoints and performed the required actions, and I wrote lots of tests. Simple, but succinct.
After I was hired
Ironically, my second or third task at Teachable was to revamp the take-home assignment. This was for a number of reasons.
- For one, the engineer who originally built the API was no longer with the company, and Teachable had lost access to the repo, so we could no longer maintain it or iron out some fairly glaring issues and anti-patterns.
- We also wanted to provide more thorough documentation for candidates to read on the mock API’s landing page. Our hope was to provide more context by displaying what the API responses would look like, how to properly query it via curl, and what response status would look like. (You know, basic API docs.) This would help minimize the need for us to answer a lot of FAQs based on some justifiable confusion.
- For another, the clunkiness of the original mock API often blocked candidates or forced them to ask lots of questions, so we were painfully aware of the weak points and wanted to avoid them in V2.
- Last, the existing take-home was a bit too simple. There wasn’t too much of a challenge built into the test. We wanted to up that a bit in order to make the difficulty level more appropriate to the positions we interview for.
Rewriting the take-home project
We’ve put a lot of thought into the new take-home and what it should be. We settled on this: write a gem or library that wraps a Teachable to-do list (Todoable, as we affectionately call it). It does what you’d expect: authenticates a user and handles some CRUD actions on lists and list items. After specing out the endpoints, we built a quick Sinatra app and deployed it. When we switched over to the new take-home, we let candidates know that they’d be beta users and encouraged them to write in with questions — especially if they caught bugs so we could make sure to address them straightaway.
Here’s a bit behind the thought process we put into the new assignment:
- Constraints: We wanted to ensure that our assignment was open-ended. If candidates write in with questions like, “Should I do this or that?”, our answer is generally, “We look forward to seeing what you come up with.” Our thinking is that we want candidates to reason about their design choices, how they test their code, what libraries (if any) they choose to incorporate, and why they’ve made the decisions they did. Rather than expecting the submissions to look one way (truly, we don’t), we look forward to reviewing what candidates turn in. Simple problems require simple solutions, and clean code is clean code, but there’s not always one correct answer, and we’re open to variety. We always make a point to ask about a candidate’s thought process in the in-person interview.
- Quirks: We built in a minor oddity or two. We did this because we want to see how candidates work around an issue, should they choose to address it.
- Complexity: Ultimately, the complexity in our assignment is holistic. We’re not just looking at how clean a candidate’s code is but also at how they’ve architected their gem or library, what their testing strategy looks like, and the design patterns they’ve utilized. We’re looking at the big picture, so to speak. In designing our take-home, we wanted to provide candidates as blank a canvas as possible to work with — we love seeing the submissions.
- Timing: We wanted to provide candidates with a challenge that we felt was manageable within a reasonable timeframe. We decided our assignment should take as few as two hours (assuming the candidate is familiar with the type of problem). Of course, with any challenge, one could spend 10 or more hours working on it, but we don’t want that. If a candidate finds they’re spending too long on the assignment, we’d welcome clarifying notes in the readme, code comments, or pending tests that explain what else they might have done.
Next steps
We’ve now been using the new take-home for a little over a month and it’s been very smooth sailing. However, we’re already thinking of building another version, or possibly even scrapping the one we currently have. Some ideas we’ve bounced around:
- Rather than asking candidates to write a gem or library that wraps an API, we could give them one we’ve written and have them write the API that makes it work.
- Have the candidate write both the gem or library and the API.
- Build in some more algorithm-heavy challenges (akin to problems candidates are given in traditional whiteboarding sessions).
We’re open to ideas and are keen to iterate upon what we have in order to ensure we get the most representative sample of a candidate’s abilities. Ultimately, that’s the goal.
Takeaway
We’re excited to see what candidates hand in to us. Given the open-ended nature of our take-home, we’ve had candidates take it upon themselves to build (and even deploy) their own APIs complete with front-ends for us to play around with. (Totally not necessary, but that kind of extra effort definitely earns some brownie points.) Our hope is that in providing minimal constraints, candidates will feel free to architect a solution that makes the most sense to them, and then tell us why.
