Making a molehill out of a mountain
3 ways an AI product elevated the user experience of UNHCR staff
When the UN Refugee Agency (UNHCR) innovates a process, the ultimate value added is an organization more primed to deliver for forcibly displaced people around the world. This is true in the efficiencies recently gained in the UNHCR’s Division of Human Resources (DHR) through the ‘ARiN’ project.
ARiN is a machine learning software, designed in collaboration between DHR, the Innovation Service, and the Division of Information Systems and Telecommunications (DIST). It supports the screening of Talent Pool applications as part of UNHCR’s recruitment process. The way it has come to serve the organization is cause for inspiration.
A turn to innovation
A few years ago now, John Thomas, Chief of Section in DHR/APRS, sat down with a number of colleagues with a common challenge on their hands. There were thousands of applications to UNHCR’s various Talent Pools, but insufficient capacity to process them. In seeking a solution, three potential paths opened up: hiring more staff, outsourcing the process, or finding a tool to relieve the workload.
Hiring more people, or sending the needs to an external company, carried a heavy price tag and was an unattractive option. John recalled, “It so happened that at that time I received an article from a professional network which talked about AI in screening processes. I shared the idea along to Julia Schtivelman-Watt, Head of Service (DHR/ATMS), and Netta Rankin, Senior Business Analyst (DHR/SPA), and we began exploring the option with the Innovation Service.”
Sofia Kyriazi, the Innovation Service’s AI Engineer, stated that the development of ARiN had to reflect the ambition and expertise of its users: “Bearing in mind the users who are all highly skilled in recruitment, the complex data they deal with on a daily basis, and the intricacies of their processes, I am proud that they were highly-demanding of new technologies and systems.”
The search for an innovative solution to support the challenges of users was at the heart of this project from the outset. The outcome was an attempt to use machine learning to accelerate this time-consuming part of the recruitment process. The story of how collaboration brought ARiN into being is explored here.
Adapting the change, adapting to change
When Zen Patel joined as Manager of the Talent Pool Management team last year, ARiN was springing back and forth between DHR and Innovation in a testing phase. An adjustment would be made. Zen’s team of four recruiters and a database manager would test it, discover bugs, and the process continued, moulding ARiN to the realities and needs of those who would eventually use it on a regular basis.
In the midst of perfecting the features of the new tool with its users at the centre, Zen highlights that implementing ARiN was not simply a case of managing the new functions and roles, but also a case of managing users’ perceptions and relationships to their tools and processes:
“There was quite a lot of change management that had to be implemented to start using ARiN properly. We worked through the challenges and the skepticism. There was a lot we had to do in terms of trying to adopt it into our everyday use, and there were debates about what a robot can or should do versus what a human can or should do.”
An enhanced user experience
So with the challenge identified, a solution developed, and change managed, what has ARiN actually achieved in terms of the user experience? How has it brought value to UNHCR?
- Faster processing of applications
Zen tells us that, “We have achieved much higher levels of screening. Prior to implementing ARiN, we were just not able to review the workload.” From verifying that degrees are from accredited universities, grading applicants according to number of years of experience and level of qualification, to highlighting key words that a suitable application should contain, ARiN processes masses of information in a short timeframe, allowing a much greater choice of applicants to be considered in the next stage of the recruitment process.
Previously, screening one application typically took anywhere from 3 to 15 minutes. With ARiN, it takes about 45 seconds.
2. Enhancing Recruiter Role
If a time-intensive process in your day-to-day work became at least four times quicker, how would that change your work? For colleagues in the Recruitment team, ARiN has meant that there is more time to interact with applicants and more time to find the right people for the right position — or ‘nurturing the Talent Pool’ in HR-speak.
What enabled ARiN to fit seamlessly into the requirements of users, Sofia contends, was strong communication between designers and users: “You need the technical teams to support and communicate their efforts effectively, as well as the users to communicate back the requirements and cognitive processes in their work.”
3. A bias against bias
ARiN was designed with diversity and inclusion at its core. Parts of applications which may indicate the identity of the applicant, such as name, gender, age and nationality, are hidden, and only the applicant ID is shown. This reduces the risk of unconscious bias, and focuses decision-making on factors like work experience, educational background, and the candidate’s expression of interest.
Sustained user testing and communication brought ARiN from quantitatively improving the screening levels, to qualitatively ensuring that the values of the organization permeate our tools and processes. In practice, diversity-sensitive design means that the machine training profiles include more excellent people of diverse backgrounds and experiences.
A new frontier for UNHCR
Director of Human Resources, Catty Bennet Sattler, praises the achievements of ARiN: “Combining Artificial Intelligence with human inputs has really been a game changer for how we do things in our organization — more efficient, more accurate, and still that human touch that’s so important.”
Catty continues, “ARiN has allowed DHR to step into a whole new area of innovation: it has propelled us into scaling up our efficiency and allowed us to focus even more of our attention on improving the employee experience for our workforce.”
The opportunities of automated software, while requiring innovative processes, specialist skill sets and ethical consideration, present a profound opportunity for UNHCR to do more with less, to relieve pressure points and backlogs, and ultimately to deliver for Persons of Concern in a fast-changing world.
