Making recruitment fair for people of colour
By Edd Sowden
Over the past 6 months in Customer Products (one of the engineering groups within the FT) we’ve been iterating on our recruitment process, again. This effort was started after a colleague challenged us on why we don’t interview many people of colour for our engineering positions and if our recruitment process was even fair to people of colour.
We’d already done work to ensure that we were both attracting and hiring with good gender diversity. This includes things like removing masculine coded words from job adverts, and reducing the length of our take home exercise (for people who may have limited time due to raising a family). Upsettingly though, we’d never investigated if those improvements also helped people of colour, or what challenges people of colour specifically might face.
Doing some research
To get to know the problem more I started looking into some of the data around hiring people of colour, and reading about improvements other people had put in place. There have been numerous academic studies done around the challenges of finding jobs as a person of colour, so there was a lot of reading that could be done.
In one study in America they tested response rates for ‘whitened’ CVs, that is where they switched candidates’ names out (for example using ‘‘L. James Smith’’ rather than ‘‘Lamar J. Smith’’) and removed racial clues from experience (for example ‘‘Aspiring African American Business Leaders’’ became ‘‘Aspiring Business Leaders’’). Through just these changes they saw response rates to the CV increase from 10% to a 25% response rate.
This is also mirrored in the UK which can be seen from a study done for the Department for Work and Pensions. It breaks the numbers down further to different industries, and also location or size of organisation. It’s worth noting from that study that when split by application type (custom form, or CV submission) using a custom form removed almost all discrimination.
So one change which we implemented quite quickly was to start anonymising all applications before a candidate arrives for a face to face interview. We get our recruitment team to do this before applications are seen by anyone in engineering.
This should cut down the bias before a candidate gets on site. But once candidates are here it’s not possible to use such techniques. So for that we needed to look at our process again and work out where other bias might enter.
Revisiting our interview scripts
One way to remove bias from an in person interview is to ensure that everyone is marking all candidates using the same criteria. In Customer Products there was a standard set of questions that were supposed to be used for interviewing, but after speaking to interviewers it turned out not everyone used all the questions and most people had a different expectation as to what the answers should be.
To ensure we had a shared understanding of what good answers looked like we decided to start again with our questions. This time, rather than just writing questions down, we started with writing down the traits we wanted candidates to demonstrate, be that team working, ability to deliver, or leadership.
With our list of traits we were then able to use our Engineering Progression framework to write examples of how an engineer at that level can demonstrate the trait. By linking our interview questions to our internal progression framework we’re able to ensure that people coming into the organisation are levelled at the same level as we judge promotions internally. Which should also help ensure that everyone is fairly levelled and compensated against their peers.
At the end of this we had a set of traits with definitions of how to meet them. From there we were able to come up with questions to allow candidates to demonstrate those traits. We made sure that all the questions were open ended and shouldn’t leave candidates feeling like there is a trick.
Changing how we conduct interviews
We now have new standardised scripts of questions to ask. Every candidate should hear the same questions and get the same opportunity to demonstrate their ability. Their answers should also be marked against the same criteria.
We created score cards to go along with the new questions with space for interviewers to write notes during the interview. We now ask interviewers to print these and fill them in during the interview. Getting interviewers to make notes ensures they aren’t then relying on their memory, which is easily influenced. The notes are used as reference after the interview when we run a review with all the interviewers to decide if we think the candidate is a good match for the job.
We’ve started asking interviewers to not speak to their fellow interviewer when the interview is over until they have written a summary of how it went along with a decision: strong hire, hire, no hire, strong no hire. We purposefully don’t have an on the fence option. By not talking to your fellow interviewer it cuts down on influence from a more senior / experienced interviewer over a less experienced interviewer.
The new scripts and scorecards have also reduced the anxiety and stress felt by our interviewers. Before, interviewers used their own interpretation of how a candidate performed, now we’re able to make evidence based calls on if a candidate succeeded. This has also helped our recruitment team give clearer feedback to unsuccessful candidates as to areas we’d feel they should work on.
Did we achieve our goals?
Our original goal was to increase the number of people of colour we were interviewing and ensure our interview process is fair for people of colour. Our recruitment team has separately been working on getting a racially diverse pool of candidates. But rather than looking at the recruitment pipeline our changes have targeted removing bias, and ensuring that people of colour who do apply get an equal chance to demonstrate their abilities, and then judge everyone using the same criteria.
One of the other goals that emerged near the beginning of this project was that any of our engineers could conduct and interview and come out with the same answer. This has the benefit of enabling us to widen the interviewer pool to our entire team. For this goal I think we’ve come a long way from where we were, and as is evident in the post interview discussions which contain good evidence based reasons for decisions.
The future
Is our process now perfect? No. We’re not claiming to have removed all bias from our interviews, and we’ve already found some of our questions could be improved, which is especially true in our technical interview. So there is now some work to iterate all the questions again now they’ve been used for a range of candidates.
We’re also questioning if we’ve gone far enough in some of our changes. For example we already get candidates to do a technical test before interviewing, so can we use the outcomes of that in place of a formal CV? As referenced in the study earlier, custom forms were very effective at reducing racial bias.
We’ve worked on ensuring people will have a fairer interview. We now want to work with our recruitment team to find ways to help to ensure we get a racially diverse candidate pool applying for our jobs.
We’ve also committed as a team to doing regular retros with everyone who’s been on an interview panel. Our goal is to keep iterating on our process as we know this isn’t done, but I hope that these changes have at least taken us in the right direction.
