Revolutionising Recruitment: A test for AI in the United Nations
By Jennifer Brookland, Independent Writer
For UNHCR’s Division of Human Resources (DHR), an idea to use Artificial Intelligence (AI) to assist with recruitment could end up revolutionising the recruitment screening processing, taking mere seconds to accomplish what currently takes staff members days or weeks. The machine assistance is part of a multi-step process to screen applicants in the agency’s talent pools. Humans still, however, validate the machine process or outcome and handle the nuances of recruitment. For now.
With staff across the agency rotating every few years and external talent pools of thousands of hopeful hires across 29 categories, recruiters spend meaningful time on each posting sifting through interested applicants, qualified or still in the vetting process. At the start of 2018, recruiters at UNHCR’s Affiliate Partnerships and Recruitment Section (APRS) were using various manually-intensive methods to screen candidates, methods both familiar and frustratingly slow.
An expedient way to comb through the masses proved elusive. Screening questions were too easy, it seemed — applicants could figure out the expected answers and pass through to the next round, and that would not help. With the current Human Resources software, Peoplesoft v9.2, a solution wasn’t coming fast enough, nor did it seem fit-for-purpose since maintaining a database of questions was also going to be time-consuming. At the same time, conversations were erupting around buying a separate system for applicant tracking or outsourcing to a 3rd party, but the associated costs and effort were a clear impetus to find another way. For Senior Business Analyst Netta Rankin, the issue at the heart of it was that her colleagues seemed to be massively labouring through their screening tasks.
“At the heart of it, I just like to help people and I don’t like to see them suffering through their jobs if I can work with them to help find solutions,” Rankin says. So, she set about doing something about it with her colleagues. This is her second time being involved in bringing new technology to DHR.
Ten years earlier, when Rankin arrived as a consultant and then new IT Officer at UNHCR, she saw how recruiters manually sifted through candidates’ applications. She was part of the team that brought the then-new PeopleSoft v8.9 system to UNHCR in 2005/6, and so she worked with the Recruitment colleagues to change things up and establish a new database format, based on which the Peoplesoft solution could be implemented as the main Human Resources (HR) tool.
Introducing the new PeopleSoft HR system and its various modules was not an easy sell at first. HR staff were used to their mainframe system. Some were convinced they did their job in a unique way that the software would not capture. They couldn’t see how the new system would help. Rankin recalls being met with a lot of crossed arms.
“Some people were pretty upset actually,” Rankin remembers. “We had resistance, and there was. Staff who were quite comfortable on the legacy system were unhappy about having to be trained on something new. In the beginning, they only saw the things that their system did really well, and failed to see all the things it didn’t do that the new system provided.”
Eventually, when it came time to work on the Recruitment module, by 2009, recruiters were ready to try the new software. Rankin, with her background in consulting, has helped companies use technology to solve problems since the late 1980s. She’s learned about getting people in conservative environments to accept tools they don’t fully understand.
“Mostly you have to demonstrate what the benefit is for them, the users,” she explains, saying pitching something new from the perspective of management or the organisation will leave staff uninspired. “You need to engage people doing day to day work and get them excited about it, and understand the benefits. You can’t expect them to just happily get on board. It’s about trying to find things that will make them feel good about receiving this change.”
So a decade later, as the once-new software she’d led to install has become both entrenched and cumbersome in some ways, Rankin was excited to champion something new. She did not expect it to be something new to all of UNHCR. But a chance conversation with one of her colleague, Julia Schtivelman-Watt, DHR’s Head of Assignments and Talent Mobilization Service, led her to speak with Hans Park, a Strategic Design and Research Manager with UNHCR’s Innovation Service, about the possibility of using Artificial Intelligence for recruitment.
A little disruption
“I thought innovation was just about coming up with new ideas, for example, tents or sanitation for our people of concern,” Rankin says. And to her, AI meant something to do with robots. But as the conversation with Park progressed, she started thinking about all the ways AI and machine learning could work for her division. And nothing she brought up seemed beyond Park’s imagination. “I was pretty excited about the fact that nothing I was saying was fazing him,” Rankin says. “Everything seemed possible.”
Focusing on the Affiliate Partnerships and Recruitment Section’s screening tasks seemed like the right first place to see how AI could change things, according to Park. “We identified a challenge we could work on together that wasn’t too difficult and wouldn’t disrupt the existing system too much,” he says.
They thought about the thousands of people on applicant lists and how they were currently being screened, with nine humans manually evaluating based on careful reading of the full application. Does Applicant A have the minimum requirements? She’s still in the running. Does she have relevant experience? No. She’s out.
“This processing seems rather linear, whereas with machines we have the luxury to work with a non-linear process of selection, meaning we can go back and forth checking data instantaneously as necessary, compared to how a human can work,” says Park. It also takes hours, days or even weeks for a human to process thousands of applications. A machine can do full processing in two or three seconds.
So together with two other members of UNHCR’s Innovation Service, Sofia Kyriazi and Rebecca Moreno Jimenez, the team built a system that takes in all the information from applicants’ work experience and letters of interest, searching for terms and analyzing language to pre-screen candidates who may fit a Talent Pool profile. Instead of spending days going through applications on a search for keywords, recruiters can spend more time in the other parts of the applicant vetting process.
With high turnover in the recruitment area of Human Resources, knowledge about how to evaluate candidates’ experience and attributes repeatedly walked out the door and had to be built up again. The machine would be able to keep it, and build on it.
Conversations around the technology centred on more than just code. Park, Rankin and the recruiting team had to grapple with questions about bias, self-awareness, and ethics.
“How do we create trust in a machine?” Park says the team members questioned. “How do the recruiters trust that the machine is doing its job correctly? Are we looking into fairness? When humans shortlist candidates, how biased is it? And when a machine does it, can we eliminate those biases, or are we creating new areas where we are not fair?”
Rankin says the system is expected to offer consistency across recruiters, reduce potential mistakes with respect to screening candidates in vs out, and especially speed up that process. Before implementation, the system will be carefully tested, and the results will be validated on an ongoing basis even after it is live. The system works much like a human brain, a recruiters’ brain that is, and so the team named it Nero to give it some form of humanness.
The cycle of scepticism
But Nero wasn’t the object of immediate affection from some colleagues. Once again, Rankin found herself faced with scepticism from those uncomfortable with the idea of a machine making screening decisions.
For those recruiters who believed their jobs were too nuanced and experience-based to be entrusted to a machine, Rankin described how Nero gets “trained” just like a new recruiter would be, with plenty of input, and rules or guidelines on how to make decisions just as a human would.
“When a new recruiter joins DHR you have to explain what to look for,” she says. “It’s the same for a machine.” She says the hard part was actually writing down what everyone is thinking as they go through that selection process. But there is a process. And therefore, it’s a process that Nero can learn or imitate.
“I think now they understand that these machines aren’t separate entities, they’re trying to mimic what we do and just do it faster,” Park says. “It’s a challenge of communication, because it doesn’t help that the machine just works,” Park says. “People want to know how and why.”
The team recognizes that human biases could be programmed into the machine. If the implementation of Nero is done by a white North American male, for instance, keyword results could reflect his implicit bias or even spelling proclivities that would favour American turns of phrase and vocabulary.
“It’s been interesting to be in these rooms and be discussing how philosophical questions become reality when they need to be hard wired into a system that impacts people,” says Park. “We don’t take it lightly. There’s a lot of scepticism and we welcome it, and are constantly trying to improve the code and the application of it.”
But the team working on Nero is focused on teaching the machine to operate with as little bias as possible. And Park thinks they’ve succeeded, for now.
Besides improving UNHCR’s efficiency and speed in looking for the right talent of people to send on posts around the globe, Park says Nero will make recruitment more fair and inclusive. “That gets into the broader (hiring) goals of inclusion, diversity and gender equity,” he says. That is the goal.
A new excitement
Future analysis will be helpful in identifying whether the machine is thinking more like a man, a woman, an African, a young person, et cetera. The team will also continue discussions about what to do if and when Nero makes a mistake- a conversation technology companies who are working on everything from driverless cars to facial recognition software, are also contending with.
“If someone loses trust in the system because it makes a mistake…will it be okay, because humans also make mistakes?” asks Park. “Or will expectations of machines be higher?”
Humans still verify Nero’s results. At least, they do currently. “We call it pre-screening in fact because humans still manage the nuances,” Rankin says. “They are not ready yet to fully trust their Nero. Maybe one day we’ll drop the human verification part, when we see it’s trained so well it’s never making a mistake. But we’ll never leave it to evolve on its own.”
For now, they’re excited to see how Nero does as it comes online for a pilot. Because the more recruiters use Nero, the smarter the machine gets. And as recruiters begin to rely on Nero to filter thousands of applicants, they’ll be able to focus on the more human-centric aspects of their jobs.
Rankin hopes this boosts job satisfaction for the humans who work in DHR. As her work incorporating AI and machine learning expands, she’s one example of a UNHCR employee with a renewed passion for her work, and a new awareness of what is possible.
“I feel a new excitement in my job,” Rankins says. “I feel like I woke up to something that I wasn’t aware of…I feel quite energized at this stage of my life that there’s this whole other world that could revolutionize the way we work.”
She also noticed that this time around, change came a bit more easily. It’s a sign of culture change within the agency, that more people are willing to be uncomfortable in their jobs and courageous, as they try new things and test out ways to improve service. This time around with this AI initiative she wasn’t pulling the team along but rather, working very collaboratively with the APRS team of recruiters, and the enthusiastic support of the Innovation Service.
“These days, people are questioning the way we do things, and looking for, and wanting technical solutions,” Rankin says. “In many implementations, I’ve done I often had the role of going out and convincing people to change. And now I feel that people are coming for change, and the machines are waiting for them with open arms.”
This essay was originally posted in the recently released publication — UNHCR Innovation Service: “Orbit 2018–2019”. The publication is a collection of insights and inspiration, where we explore the most recent innovations in the humanitarian sector, and opportunities to discover the current reading of innovation that is shaping the future of how we respond to complex challenges. From building trust for artificial intelligence, to creating a culture for innovating bureaucratic institutions and using stories to explore the future of displacement — we offer a glance at the current state of innovation in the humanitarian sector. You can download the full publication here. And if you have a story about innovation you want to tell (the good, the bad, and everything in between) — email: innovation@unhcr.org.
