Our journey to refine our developer hiring process

Developer hiring documentation

Context

In a previous article, I explained how improving your talent assessment and job definition can significantly impact talent acquisition, without excessive investment, using one of our case studies as an example.

In this article, I will focus more on the low-level details to explain precisely how we refined our recruitment process and the key milestones to achieve the mentioned results.

Step 1 : kick-off the project

One of the most important decisions was to find the right tech partner and involve them as a co-owner of this project. At this moment, we recently welcomed our VP of Engineering who will be responsible for managing all of our tech recruitments.

Our current pains and performance mentioned in the previous article made it easy to present the problems we were facing and how impactful it will be to our business if we don’t hire faster. I could easily switch to the solution by explaining the benefits of the job scorecard and structured interviews from the A-method, which is one of the most famous methodologies in recruitment known by many leaders.

Finally, your tech partner can easily find volunteers to create this working group. In our situation, we worked with collaborators from different levels, including 2 developers, 1 Engineering Manager, our VP of Engineering, and myself as the recruiter.

Step 2 : frame it

Our primary objective was to reduce the time it takes to fill a developer position. To achieve this, we aimed to:

• Improve our job definition and skill assessment
• Streamline our hiring process

We had also a few constraints to consider, such as:

• Maintaining a good candidate experience,
• Keeping an efficient and reactive hiring process if we need to speed up the process for a candidate
• Upholding high-standard expectations

Then, we decided to define our key metrics that will help us measure the success of our initiative.

Our north star metric was clearly the time-to-fill a developer position.

To make sure we’ll see improvement in our skill assessment and job definition, we decided to follow these metrics:

• Quality of hire: if your assessment and job definition are reliable, you should be satisfied with your new employees and significantly reduce hiring errors.
• Estimated vs actual level: if you notice a difference between the perceived level during interviews and the actual impact of your new employee, you may have potentially made errors of judgment during the hiring process.
• Candidates per hire: The number of candidates that the hiring manager interviews before a position is filled should decrease. This is because you will have a better selection in the previous steps, such as screening and focused interviews.

To ensure that we scale and streamline our hiring process, we selected the following metrics

• Number of interviewers: obviously, if you are able to include more interviewers, your process will be more scalable.
• Percentage of the tech team ready to interview: by increasing the percentage, the hiring process becomes more efficient. Multiple interviewers reduce bottlenecks and speed up candidate evaluation. This metric also distributes workload and prevents overload on a single interviewer.

We also identified other relevant metrics to review to measure the impact:

• Drop rate: developers are more likely to be engaged if you provide well-structured assessment methods, as 31,6% of developers dislike technical questions unrelated to the role.
• Closing rate: by showing a clear process and using effective assessment methods, you can improve the chances of candidates being convinced to join your organization.
• Cost per hire: According to a survey, 80% of developers share their hiring experiences with their peers. Therefore, having a clear understanding of what you’re looking for, being able to explain why, and knowing how to assess the specific competencies you need can lead to a more effective inbound funnel in the long term, resulting in reduced expenses.
• Effort in talent acquisition: If your hiring team spends less energy on the search, it demonstrates the effectiveness of your hiring process.
• Time-to-hire: while average data provides the trend, it’s also important to understand our best hiring speed to ensure we maintain the efficient and responsive process we desire.
• Candidate satisfaction: to ensure we maintain or improve our candidate experience

Finally, we built a 6-month retro-planning that we adapted based on the available resources, depending on the sprints and product roadmap. This retro-planning was composed of 5 next steps, mentioned below.

Step 3 : define the need through job scorecards

Our goal was to create reusable scorecards for mid- and senior-level developers, as we already knew they would be our top priority levels for the following years of recruitment. These scorecards would serve as a tool for managers and new employees to align their expectations on performance and impact. The scorecards needed to be clear and measurable, specifying the expected outcomes for different milestones (3 months, 6 months, and a year). They also needed to be aligned with our career ladder to ensure consistency in supporting new developers for the long-term.

To achieve this, we followed the A-method structure with the group. This ensured that the outcomes would align with the company objectives. Then, we defined the most important competencies required to meet these outcomes, along with the expected behaviors associated with these competencies. We started with a launch workshop where I explained the value of scorecards and how to build them. Then, we moved on to the production phase, where the working group collaborated to create the scorecards. Finally, we concluded with a big final session where everyone had to present their own scorecard to collect diverse opinions and build a unified one.

In the end, we successfully established a standardized and comprehensive framework to evaluate the skills and expectations of mid and senior-level developers.

Step 4 : review and adapt your hiring process

Our actions

In this step, we set several objectives.

• Understand the current interview process, identifying its strengths and weaknesses, to determine what to retain and modify.
• Gain insight into the experience of new interviewers during their training program to enhance their learning journey
• Define a new hiring process aligned with the job scorecard’s required skills, ensuring its specificity for the position.

At this time, our primary strategy was to preserve developers’ resources by preparing workshops, collecting and presenting insights from interviewers and employees (as candidate feedback) with my tech partner. This was followed by a workshop where we all defined the hiring process steps and align them with the identified skills.

Our results

Regarding our hiring process, one of our main concerns was related to an exercise that did not have a linear difficulty level. We were also not confident that it accurately assesses the required skills. Another concern was the number of blocking steps, as each interview was followed by a decision process. Consequently, the entire process generally took too much time, often exceeding a month. Additionally, we noticed repetitive HR questions during our technical exercises, which prevented us from delving deeper into candidates’ career aspirations and motivations.

For the new interviewers, they primarily mentioned a lack of guidance and structure regarding our needs, requirements, and interviews. Some of them also expressed a lack of confidence in assessing candidates with different programming languages, as proficiency in a specific language is not a requirement for our ideal developer.

That is why our main solutions were to:

• Implement a Topgrading interview to replace the HR questions and answers in our current process during the technical interviews.
• Implement a decision meeting only after the candidate performs both technical interviews. This will give us a better overview of the profile after making an informed decision
• Entirely redesign our exercise to adapt to the required skills and seniority
• Create an engineering hiring toolbox that provides guidance and resources for interviewers and hiring managers who are looking to hire tech team members
• Create an interviewer availability based on interview type and expertise

Step 5 : deep dive into your interviews

Our goal was to define the main steps within each interview. This involved establishing a balance between structured and non-structured questions for each step, as well as determining the set of structured questions to be asked. We also explained how the interview would be conducted and how to assess candidate answers.

To accomplish this, we assigned each group member a specific step based on their expertise. Developers were responsible for defining focused interviews, managers handled topgrading, and I took charge of screening. We then conducted a second workshop to present and validate the new process, ensuring that it aligned with our objectives, constraints, and covered all the necessary skills.

Step 6 : add support

Support tools are essential as they provide clear guidelines, improve team collaboration, enhance the candidate experience, and increase overall efficiency.

That is why our goals were to:

• Establish clear evaluation criteria: this involved determining the scoring system and interview kit format in order to make informed decisions.
• Build our interview coordination to ensure effective communication with candidates and interviewers. We also outlined all the actions to be taken from pre-screening interview to closing, both orally and in writing, to ensure a smooth and efficient hiring process.
• Define and build our training program for new interviewers, combining theory and practice, to prepare them to be official interviewers.

Regarding the format, if you need to save resources from the developers, this is where you can have a strong impact and take ownership with your tech partner.

In our situation, for both the training program and evaluation criteria, we valued the collaboration between tech partner and the recruiter.
On the other hand, we decided that I could have have the full ownership of talent coordination. Owners work on different options with a decision on their side and take the time to present it to the rest of the group in a sync meeting for validation.

In terms of results:

• Our evaluation relies on a scoring system ranging from 1 to 5. We use interview kits in our ATS TeamTailor to centralize all the information.

• Our coordination documentation can be found on our Notion page, where everyone can find the oral explanations and actions to conduct.

• Our training program combines theory and practice. It includes:
  1. General hiring knowledge from the recruiter, where I delve into their recruitment experience, share our company’s hiring guidelines, present our ATS, and communicate my expectations as a recruiter.
  2. Tech hiring knowledge from the tech partner, who explains the role of the interviewer, how to prepare, conduct, and follow-up for a specific interview.
  3. Practice sessions, which consist of 2 shadowing and 2 reverse shadowing exercises (additional sessions are possible if needed), with ongoing discussions with the trainer / official interviewer.

Final step : test, measure, review and iterate

Now is the time to test this new hiring process and collect insights.

For the first few months, we continuously asked for qualitative feedback from interviewers after interviews and from candidates during follow-up calls to identify quick improvements.

For the quantitative data, we decided to wait a full year to analyze the results so that we would have a larger sample size, as we were not hiring 15 developers in a quarter.

After gathering this information, we reviewed the results to understand what worked well and what didn’t. The real-world data and feedback helped us understand that we made significant improvements across all metrics. That is why we’ve chosen to standardize this initiative across all departments, rather than focusing solely on improving this one.

Thanks to this initiative, hiring a developer at Shipup is now a more pleasant task.

➡️ The impact of this refinement

To sum up

Refining our developer hiring process was an insightful journey.

It helped us understand where we excelled, where we needed to improve to achieve better results, who our ideal developer is, and what we expect from them at different timeframes.

We also worked collaboratively across different levels to accomplish this. This brought diverse perspectives to the table, leading to more innovative solutions and better decision-making. It also promoted a sense of unity and shared responsibility, as everyone involved had a stake in the outcome. Finally, it fostered a more inclusive and transparent work environment, as it allowed for a more comprehensive understanding of the challenges and opportunities at hand.

The best part of this project, from my perspective as a recruiter, was the feeling I got when I didn’t need to intervene, and I could watch the group members think, debate, and find the best decision by themselves.
In our workshops, we talked, argued, and shared different views, which was fascinating. These discussions helped us see things differently from both the interviewer’s and candidate’s points of view.

Here are my 5 key takeaways to make this project a success:

1. Involving your tech partner as a co-owner will greatly impact this initiative. Such collaboration encourages shared responsibility for the project and ensures a balanced perspective from both recruitment and technical aspects.
2. To achieve the desired outcomes, define clear objectives and understand your constraints related to the existing process, time, resources and more. These will guide the process and shape the strategy.
3. Establishing clear and measurable job scorecards helps align expectations regarding behaviors and impact. You should signifcantly cut down the time spent on alignment, which is often the most challenging part of the recruitment process.
4. Allocate resources from working group members only for major topics, such as brainstorming and decision-making sessions. Discuss project planning and management only with your tech partner. Your timeframe for this project should significantly reduce as it will less likely impact the product roadmap or the business.
5. Lastly, gather both qualitative and quantitative data after the launch. This will help identify what works and what doesn’t, and make adjustments as necessary. This will also help you draw conclusions and prove the ROI of this investment to top management.

If you are interested in improving one of your hiring processes, feel free to contact me on LinkedIn. I would be happy to discuss it with you. I am also eager to hear about your reading experience and how this initiative can help address your specific challenges.

If you are interested in joining our team, I encourage you to learn more about the available positions by visiting our career page here. We look forward to hearing from you and exploring the possibility of having you on board to help us achieve our vision.