ENGINEERING
How we doubled Canva’s engineering growth rate in 6 months
How we overcame a talent shortage to deliver on an ambitious product roadmap
A company’s ability to ship product ultimately lies in the ability of the recruitment function to support the team with new engineering talent. But what do you do when you’re faced with a talent shortage, and a company that has an ambitious product roadmap? Head of Talent Acquisition at Canva, Rick Goodfellow reveals how the team introduced a few changes to its recruitment process to help meet this demand.
It’s a universal problem I see every in-house recruiter face when trying to scale up an engineering team. When a company like Canva has an ambitious product roll-out plan, it’s no longer a question of how it can hire more engineers to ship product — the challenge is how to make more hires quickly, without compromising on skill and capability or negatively impacting other parts of the business.
So while it can be tempting to bring in more recruiters in a bid to unlock the candidate pool, the key is to first look into optimizing the effectiveness of one’s current hiring process — which has absolutely nothing to do with the size, or the ability, of our talent acquisition team.
This is what I was brought in to do at Canva when I joined the team earlier this year. Having helped build Google Australia’s engineering team from scratch, there were some lessons I was able to bring in to my new role, starting with how we could improve our recruitment process for engineers.
There were a couple of areas we needed to work on to be able to scale to the size we needed to and reduce the amount of time that our engineers spent interviewing per hire, namely:
- How to refine the process to get better candidates into the pipeline
- Supporting our engineers throughout the interview process
In this blog we’ll explain how Canva overcome these challenges and increased the chances of having more of our candidates pass our stringent interview process and receive a job offer.
The challenge
As mentioned earlier, there were many factors at play that were stifling our hiring rate — and they had little to do with the size or the ability of our talent acquisition team.
In fact, when we took a step back and crunched the numbers, what we found was approximately only one in 15 candidates would eventually pass all the interviews to receive a job offer from Canva. In addition, we were losing 500 candidates a year through a fall-out during the interview process, where candidates found themselves changing their mind and deciding not to pursue this opportunity due to the time taken to complete our process. What this equates to is a loss of at least a minimum of 33 potential hires each year, which signifies a leak somewhere that needed to be fixed as this would be one of the most straightforward ways to revitalize our candidate pool.
Why was only 1 in 15 candidates passing our final interviews? It’s hard assessing a technical skill-set from a CV, so our recruiters were being asked to jump through an almost impossible hoop. Without the experience to technically screen we were relying on technical skills to be tested later in our process, taking up lots of time for our engineers who had to interview everyone.
So first and foremost, we needed to put only the best candidates through the pipeline. We did this by:
A) Introducing a technical pre-screen, weighting the qualification of candidates more towards the recruiter.
B) Introducing another option instead of the take-home challenge (which took on average 5 hrs to complete)
A) Introducing a technical pre-screen
We decided to introduce a technical interview conducted by a recruiter, designed to gauge whether a candidate had the proper fundamental computer science knowledge. This would enable us to put higher quality candidates into the interview pipeline. As a result, the successful candidate would have a significantly better chance of performing well throughout the rest of the process and the probability of a job offer would increase.
Phase 1: Getting buy-in from the team
Revamping our screening process was going to require lots of work from everyone at Canva — our engineering teams, our interviewers, and our talent acquisition team. That meant it was important to explain the rationale behind the changes, and get buy-in from people to help us make the change. For us, that meant showing how this change would provide a benefit for each group, leading to:
- less time spent interviewing per hire;
- ensuring a higher quality of candidates put through to interview;
- enabling us to have more available interviewers; and
- ultimately, less time interviewing per hire!
Crunching the numbers, we worked out that if recruiters could screen out approximately 40% of candidates we would previously have put into the interview process, we could potentially save up to 600 hours engineering hours spent interviewing per year from the recruiter screen alone.
We met with all the different engineering functions at Canva separately to show them the issue at hand, and how introducing a recruiter technical screen will help them hire better — then, we asked for their help.
Phase 2: building a platform for skills assessment
One of the worst things we could do would be to rely on subjective data to qualify applicants. As with many startups, our initial candidate pool was created through word of mouth. Someone would recommend a friend for an interview, others would hear about job openings through the grapevine. But what was missing was a strategic approach to identify a set of skills criteria for the roles we were looking to hire, enabling us to create questions to gather the data points required to make a qualified assessment.
It was therefore important to ask ourselves — what skills do we need, and how do we test for them? With this in mind, together with the engineering team we drafted an assessment criteria for each engineering role, designed to reflect the company’s needs and priorities, leaving any second guessing behind.
What this meant was we were no longer comparing whether one candidate was as good as, or even better, than our latest hire. Not only was this subjective — we were not relying enough on data points to assess a candidate’s ability to meet specific criteria for the role, encouraging us to hire people similar to who we have hired previously — we were also missing out on diversity of opinions and approach, and the opportunity to bolster on our engineering skill set, which is crucial for growth. We needed to assess candidates against a documented set criteria for each individual role in engineering. This is one way we can help minimize bias-conscious or unconscious-when deciding whether or not to hire at the decision meeting.
It’s a problem many high growth startups face. In our desire to produce and iterate, people could be brought on with minimal formal assessment — a lot of the review process was based on attribution or gut feel. The qualification of candidates relied too much on the opinion of the interviewer, and not enough on data to show how the candidate met the criteria for the role. Unfortunately, it’s virtually impossible to collect evidence of unconscious bias, but companies can put a process in place to discourage this from happening — and one way is to simply agree on the assessment criteria beforehand.
A well-defined list of set criteria helps us better qualify the right people for the role and reduce potential bias in the process.
Phase 3: drafting the questions
The next step was to draft a series of questions that could be used to assess candidates. Together with Joel Hynoski, who looks after the growth and culture of Canva’s engineering function, we set up project groups to work on various sets of questions for different areas of engineering here at Canva.
When preparing questions, it was important to consider the following:
- Prescreen questions shouldn’t be pulled off the Web. It’s important to make them relevant and tailored to the job function and your organization.
- Each role will require their own unique set of questions. In our case, that meant a list of 8 questions for Digital Design front-end, Digital Design back-end, Front-end Product, Back-end Product, iOS, Android, and Quality Assurance.
- All questions will need to be vetted with engineering leadership to ensure alignment on what they are designed to test for. Questions will need to have clear answers that a non-engineer could learn how to interpret.
Because the prescreen questions were created to require a clear and well-defined answer, it was therefore possible to teach someone, regardless of their engineering experience, to understand what a good answer would look like.
Phase 4: Tracking responses, and analyzing them
The next step was adding all the questions to Canva’s applicant tracking system so we could track candidate’s responses to various questions. From this data we could predict the percentage of candidates that will thrive in the next stage of the recruitment process, and anticipate their pass rates in the final interview, enabling us to better forecast growth.
What this has also allowed us to do is validate each question accordingly.
Example: Say you have 8 prescreen questions curated for a BackEnd role. If 100 out of 100 engineers were able to answer question 7 correctly — then there is no point in having that question in the list, so it would need to be replaced. On the flip side, if we could identify a correlation between the people getting questions 2, 3, 6 and 8 correct, and then being offered a job, we could then determine the probability of a candidate getting through to the final interview stage based on their answers to questions 2, 3, 6 and 8.
The result?
By introducing a prescreen step, we’ve been able to screen out candidates that would otherwise have been eliminated at a later stage, and by introducing set assessment criteria for each role we can be more confident of our decision when offering or rejecting candidates.
The main point of this exercise however was to reduce the time engineers spend interviewing per hire, so that we could scale at the desired rate without negatively impacting the development of our product. Due to the recruiters being able to put higher quality candidates into the process, andthe assessment criteria being in alignment at each stage, the pass through rate to offer at the most time consuming stage, the final interview, dramatically improved. This reduced the engineering time spent interviewing per hire. So far we have managed to reduce engineering time spent interviewing per hire down to 20 hours compared to 30 hours last year. This saves 1000 hours of engineering time over 100 hires, enabling us to scale at the desired rate and minimize any negative effect on the growth of the product itself.
B) Introducing an option for the take-home challenge
Traditionally, we would screen hundreds and hundreds of resumes, and whoever looked good on paper would be invited in for a chat. That was a lot of time spent on spruiking Canva, as most candidates were already in comfortable positions with another company. Despite Canva being a reputable tech startup worldwide, it was still a hurdle to get candidates onboard with our recruitment process. A large part of the difficulty was the need to persuade every candidate to embark on a take-home challenge, which often took 5 hours to complete in their spare time, just for a chance to move on to a final interview onsite.
In fact, this is where 40% of our engineering candidates dropped out. We had to take a step back, and think about whether a 5 hour challenge was just too much for some. They could be busy with kids or a hobby, or just too stressed with their current job to be able to dedicate the time.
This hypothesis was put to the test. We introduced another option for candidates — they were now able to choose between having a take-home challenge, which they could do in their own time, or a live coding interview with an engineer over a shared live environment. This interview would only take an hour, but would test for exactly the same criteria. If this worked, more candidates would be able to continue with the interview process and the probability of having more great candidates pass through to the final stage of interviews would increase.
Phase 1: Set expectations
Because we were introducing a new concept to a very traditional challenge, we needed to demonstrate how introducing an additional option to the take-home task would potentially increase our candidate pipeline by 40%. Once we were aligned, we were able to create a new live coding challenge that would assess the same skills as the take home version.
The difference between a take-home challenge and a live coding interview lies in a person’s ability to solve an engineering problem under two different environments: a candidate could choose to peruse and solve the problem at their own pace, or decide to try and tackle the same engineering problem with the interviewer watching you refine your answer — mistakes and all.
Phase 2: Feedback to reiterate
It was important to revisit the performance of these challenges to ensure every question was doing what it was designed to do.
Based on feedback from our candidates, one of the options we implemented was to allow a candidate to code in their own preferred integrated development environment (IDE), rather than one we prescribed. This was particularly important in a live coding interview, as the candidate would be inclined to perform much better in a time restricted test using their own tools, as opposed to one they were not necessarily fluent in.
The result?
Introducing this additional option resulted in surprisingly positive results. The team managed to reduce the dropout rate in our interview process from 40% to 0%. Literally every candidate we had spoken to chose to participate in one challenge or the other.
But more interestingly, the team found that there was an even split between candidates who decided to choose the take home challenge and those who opted for the live coding interview.
By accident, we realized our process was now accommodating the different ways people like to solve problems. Some people prefer to take their time and work on their own, whilst others like to collaborate in real-time. The new process we introduced was able to cater to different backgrounds to suit a diverse range of working styles.
At the end of the day, we want to ensure we’re providing a great experience for every single candidate that comes through our recruitment funnel, and accommodating for different perspectives and approaches to problem solving.
2. Supporting our engineers throughout the interview process
It was great to see 40% more candidates entering the pipeline, however even though we have reduced engineering time spent per hire, specific engineers were still conducting the majority of the interviews. We needed a more structured approach to train up new interviewers to be able to spread the workload.
Interviewing at an optimal level
For us, interviewing at an optimal level means we’re screening out unsuitable candidates as early as possible before they reach the most time consuming parts of the process, enabling a better pass through rate at the final stage.
However we still needed to ensure that particular interviewers, including recruiters, were not being overloaded- defeating the purpose of what we were trying to do in the first place, i.e. saving them time.
With this in mind, we put in place two immediate changes that would help all engineers interview at an optimal level.
A) Train more engineers on how to interview, to ensure more engineers are put through interview training, enabling us to spread the load.
B) Free time up for the recruiters to be able to screen more people.
A) Training more engineers
We used to train engineers how to interview on a very sporadic basis, resulting in an increased burden on engineers who were more experienced. Today, our engineers are trained in batches using a defined set of criteria and coursework to ensure they are fully equipped to interview effectively at every stage. Every engineer is also required to participate in a minimum of two shadow interview sessions to give them real-life experience.
B) Free time up for the recruiters
To help optimize the team’s time, we made the decision to bring in a scheduling assistant. Our assistant was able to take a lot of the scheduling and administration work from our recruiters, so the talent acquisition team could focus on identifying the right candidates.
Some of our assistant’s responsibilities include ensuring our engineers are completing all their training and shadowing sessions in batches, so that we can have 20–30 engineers trained up to speed each month. Since our assistant joined, we’ve been able to give the recruiters 35 hours back per week to find and technically screen candidates.
What does the future hold for Canva’s talent acquisition team?
These new recruitment processes have been a game changer for Canva. It didn’t happen overnight, but the effects have been significant and worth replicating. In 3 months alone, we have doubled the number of engineers that we’ve been able to hire compared to last year without adding more recruiters. Only time will tell if this hiring spike will continue, but with these new and improved recruitment initiatives and the traction that Canva is experiencing on all fronts in product and market growth, all arrows are pointing up.
Key hiring tips
Here are some tips and things to consider if you find yourself in a similar position of trying to grow your team quickly.
Set up the right system. As you scale, you need to reinvent your systems. What worked for 100 people won’t work for 300 people. A tricky part of scaling is stepping things up without just throwing more people at a problem.
Align motivations with your recruiters. Our recruiters here at Canva are invested in the company. Having people who care deeply about who we bring into our team, and really striving to build one of the world’s best companies, means we’re all on the same page.
Recruiting is a team sport. To hire, you need everyone involved:the founders, engineers, HR etc. Making sure everyone recognizes the importance of hiring and helps to build your system is key.
Spend everyone’s time wisely. Saving the team’s time ultimately means we can build more great features for our community. If we can save ten hours of engineering time per hire, that has big flow on effects and allows the product to scale.
Don’t let your candidate experience slip. Everyone who goes through a hiring process has family, friends and a network of past colleagues. A bad experience spreads, so does a good one. Even when someone isn’t the right fit right now, they could be in the future, so treat everyone with care and make sure they are supported through the hiring process.
For more details on our current job openings, visit Canva’s careers page here . Know someone who would benefit from reading this article? Why don’t you share it with them here:
Originally published at https://product.canva.com on September 5, 2018.
