How Sandbox Recruits
Building great software is hard. Building an organization that builds great software is even harder. We want Sandbox to outlast us and grow beyond us. We want to visit Sandbox in 5 years and meet a team of people we’ve never met, yet one that carries forward the spirit and mission of Sandbox. This is why culture, values, and process is so important to us, and recruiting the right people might be the most important of all.
Below is a version of the manual we used internally to guide our recruiting process in the fall of 2019. We’re sharing it publicly now to help students applying to Sandbox put their best foot forward, as well as give a glimpse into how Sandbox is run.
Learning and Iterating on Our Process
The very first semester we did interviews, we chatted with our candidates for around half an hour, didn’t do a technical challenge, and had very few standardized questions. When it came time to pick who made the cut and who didn’t, we found ourselves lost in a sea of facts about candidates, gut feelings, and inconsistent notes. Although everyone we ended up recruiting was amazing, we knew our process had to change.
By reviewing the pitfalls in our process and researching what companies like Atlassian and Medium were doing, we identified goals for our recruiting process:
- All applicants need to receive the same fair and standardized process.
- Decisions should be made objectively, and the impact of unconscious bias must be reduced.
- Doing well in our recruiting process must be reflective of doing well as a member of Sandbox.
- We need to help candidates put their best foot forward.
- Candidates should learn about Sandbox, just as we learn about them.
These goals led us to develop a set of core characteristics that we think define a successful Sandbox member, as well as write a rubric around those characteristics to help us grade candidates consistently and objectively. The rubric was condensed into a Google Form, which is then filled out at the end of each interview. Each characteristic receives both a numerical score and a couple sentences of objective evidence, or is left entirely blank.
Additionally, we created a targeted, standard set of interview questions and a technical challenge that we hoped would give candidates the opportunity to demonstrate how they match the characteristics we look for.
What We Look For
To identify our core characteristics we thought broadly about what makes a successful Sandbox member:
- Are they an effective teammate and developer?
- Do they align with our values and mission?
- Are they invested in the Sandbox community?
- Do they bring new perspectives/skills to Sandbox?
From each of the categories, we pulled out a set of specific characteristics to grade for in our rubric.
👩💻Effective Teammate and Developer
We aim to get a mix of experienced and inexperienced members. Part of our mission is to provide the right environment for student developers to grow, and we think mixing experience levels promotes a culture of Learning Together. We also prefer to get people earlier in their college career, to leave time for them to grow into Lead and Eboard roles. As such, we don’t just look for the fourth-year with a million side projects and a co-op at HubSpot. Instead, we look for base skills that we think make a successful project member.
- Problem Solving: Developers need to be able to formulate solutions to problems.
- Communication: Good development is useless if it’s not in sync with the team.
- Collaboration: Teamwork needs to be valued at every level.
- Autonomy: We love pair programming, but developers need to search for solutions before asking for help, and make some choices on their own.
- Awareness: Developers need awareness and self-reflection to be able to grow and fix their mistakes.
🙏Value Alignment
Our values are to Be Kind, Learn Together, and Make an Impact. Our mission is to unleash the power of software for researchers and students. This should resonate with members, and we need to see that in candidates too. Preferably, candidates can connect our values back to their own life experiences. We screen for each of the traits.
🏃♂️Investment in Sandbox
Sandbox is a 10 hour/week commitment. This is a commitment we make to our clients and to one another. No matter how great a candidate may be, they will not get much out of Sandbox without being willing to commit a significant amount of time. Thus, we look for two traits.
- Interest: They should be excited and invested in joining Sandbox.
- Commitment: They need to be reasonably available and prepared to commit to Sandbox.
🤯Perspective
Sandbox thrives on fresh perspectives and skill sets. Less experienced members can bring new ideas and help us reason from first principles, just as experienced members can bring knowledge and act as mentors. We believe creating balanced teams helps us build better products, and taking members with differing perspectives, ideas, and skill sets can help make us all more effective.
What We Don’t Look For
We want recruiting to be as fair, free of bias, and effective as possible. We all have biases though, and that’s normal, as long as we work to mitigate them. Interviewing is subjective, and often we can get gut feelings. However, we stick to recording the observable facts.
For example, “Kate seemed like a kind person” is unhelpful and prone to bias.
“Kate described how using kinder language as a TA helped her students absorb the content better” is useful information that helps us build an objective image of who Kate is.
Of course, this type of concrete evidence will be harder to come by. When we don’t have enough evidence for a category, we just leave it blank. Making a guess is unfair to the candidate, prone to bias, and ineffective for our recruiting process.
The Rubric
Our scoring system has five scores: “Strong No,” “No,” “Mixed,” “Yes,” and “Strong Yes.” It tends to be quite hard to get a “Strong Yes” in more than a couple dimensions. The rubric contains guidance for when to give which score for each category, which helps us stay consistent across interviewers, as well as force us to be as objective as possible.
For example, the row on our rubric for Make an Impact reads:
- Strong No: Just here for the resume. Doesn’t understand the value of impact.
- No: The candidates understands impact but does not desire to make an impact.
- Mixed: The candidate is not opposed to making an impact, but is primarily looking for interesting things to do.
- Yes: The candidate can describe instances where they made an impact on others. Making things better resonates with them.
- Strong Yes: They are excited to build real things and describe dissatisfaction at building useless projects. They aim to leave things better than the way they found them.
Lastly, we have a final Overall category that splits candidates by “No,” “Mixed,” and “Yes,” where a “Yes” is someone we’d want to take on the spot, a “Mixed” is someone we want to review after all interviews are done, and “No” is someone we likely wouldn’t go back to review.
Making Decisions
After all interviews were done, we used the grades and notes written by interviewers to build a holistic picture of each candidate and match their skill sets, interests, values, experience, and more to create a balanced incoming class of Sandbox members that can meet the current and future needs of the organization. With over 100 applicants this past cycle, we had many applicants that we really wanted to accept, but we simply did not have space for, and it was incredibly difficult to make that final choice. However, with the rubric scores and objective evidence collected, we felt a lot more confident that those that we accepted would be incredible additions to the team.
Room To Grow
After a semester in practice, our new recruiting system is an obvious improvement over our previous ad-hoc methods. It gave us a higher confidence level for each of our candidates, reduced the effects of unconscious bias, and tested for skills relevant to succeeding in Sandbox. However, based on anonymous feedback collected from applicants, coupled with our own observations, we see a range of improvement opportunities. For our next recruiting cycle, we aim to have better transparency into our process and criteria, more difficult technical challenges, and a set of behavioral questions that give candidates more opportunities to demonstrate the characteristics we look for.
