From pipe dream to pipeline: UNHCR learns from mainstreaming an AI product

Innovation seems like the obvious place to turn to when an old process is proving burdensome. Out with the old and in with the new. The initial proposition is equally straightforward: why use a bicycle when there are motorbikes out there? Not so straightforward, is that somewhere between the old being ousted and the new becoming normal is much learning and collaboration. As the application screening software ‘ARiN’ has its debut as a fully supported corporate product, we look back on the experience.

The ARiN project in the UN Refugee Agency (UNHCR) is testament both to the skills and qualities needed to realise the possibilities of new tools, and to the rewards of dynamic adaptation and collaboration in the face of challenges. This article explores the journey from challenge to a productionised artificial intelligence product in UNHCR.

Who is ARiN?

ARiN is a software solution that uses machine learning techniques. More specifically, ARiN screens applications submitted to UNHCR’s human resources talent pool for job vacancies according to a set criteria. It assists the pre-screening phase, where the recruiters make the first parsing of the thousands of talent pool applications. How does it do such a thing? Machine learning techniques mean that an algorithm is trained with input and output data until it is able to accurately predict outcomes by itself.

In ARiN’s case, educational background, years of experience, description of work experience and letters of interest are analysed and categorized. The manner in which this has elevated the user experience for UNHCR colleagues is outlined in this second article. Suffice to say for now, the fruits of this project are in the crucial time resources that ARiN unlocks for its users in their day-to-day work, while minimising bias and error.

In bringing such a benefit to bear, a collaboration between UNHCR’s Division of Human Resources (DHR), Division of Information Systems and Telecommunications (DIST), and Innovation Service was key. To understand how a solution like ARiN came from the fringes to the institutionalized center of UNHCR, we must dive into the dynamics of these teams’ collaboration, and the process that brought ARiN into being.

The Birth of ARiN

When Netta Rankin, Senior Business Analyst in DHR, approached the Innovation Service, she wasn’t sure what the outcome of their conversation would be, not least a fully integrated AI solution two years down the line. “There were thousands of applications and there just wasn’t a quick enough way to process them. We explored other options, like adapting the human resources tool ‘PeopleSoft’, but our needs were too complex for that. We thought, the Innovation Service, maybe they know something.”

Several conversations and research phases later, a simulation of the tool was developed. Sofia Kyriazi, the Innovation Service’s AI Engineer, reflects, “While we were still adjusting objectives and approaches, we were already really excited about the possibilities of machine learning to relieve pressure from the talent pool screening process.”

With support for the initial exploration secured from the leaders of DHR and Innovation Service, the design phase began and developers were brought into the fold. Over the course of last year, user testing and continual iteration brought ARiN closer to completion. Sofia continued, “From structuring the data to the user-friendly interface design of a solution, our roles as technical advisors, developers and innovators were changing all the time.” The outlook of Netta and Sofia in these early stages of ARiN, formerly called Nero, can be found here.

From Innovation to mainstream

The relationship between DHR and the Innovation Service allowed ARiN to develop in a way that responded to the bespoke needs of DHR in an innovative and experimental process. As the initial proof of concept was created by Innovation, outside of the corporate IT environment, DHR and the Innovation team then united efforts with DIST to bring ARiN into the fold.

Carlo Barrientos, Senior Business Analyst in DIST, was intrigued when ARiN landed on his desk. He remarked, “Innovation is one thing, but mainstreaming is another.” Despite a successful proof of concept, multiple issues arose around the technology being further developed. The question of the long-term support for the tool needed to be carefully considered. On top of that, the idea of a system that learns from the users raised issues of accountability and user identity management. Carlo went on, “We were joking in the office about those doomsday movies, when AI comes alive and takes over!”

Normally, when a change is made to a system, users are logged in with a username and password, so there is a record and accountability. An automated software can’t be exempt from auditing. Jointly, DIST, DHR, and Innovation team created a support model for ARiN to work alongside its human colleagues.

Netta from DHR senses a change in how new products are being introduced into the organization. “I think collaboration is key. It’s not like the old days where IT took care of everything and introduced the new products to a more technology-resistant workforce. Now we have users who want technology, and they’re coming fast with many ideas and solutions that they’ve heard about. We have to communicate and understand each other’s sides.”

In the final stage of ARiN’s productionization, UNHCR is working with the United Nations International Computing Centre (UNICC) who will take charge of its maintenance. This key partner allows for a sustained maintenance capacity, so that ARiN can continue to enhance the work of colleagues in Human Resources.

Lessons from the ARiN experience

ARiN suggests that innovation can play an exciting role in cumbersome and repetitive activities. This alleviates precious time resources for colleagues, and it can have the secondary effect of reimagining and optimizing ‘human’ roles.

In order for an organization to incubate novel technologies and products, the ARiN experience teaches us a number of lessons. With new ideas for solutions bubbling up from all corners of the organization, the need for specialized capacity is clear: business analysts, data engineers, user experience and user interface designers, and access to developers will help UNHCR to assess viability, test new models and promote innovation with the relevant expertise.

In this project, the value of partnerships is in plain sight. Earlier collaboration with the IT department, DIST, may well have enhanced the final ‘productionization’ stage by incorporating such considerations earlier in the development process. Learning from teams and organizations that have experience in implementing novel systems saves valuable time and adds much value to the product being designed.

ARiN is now ready to become a part of the ecosystem. It is the hope of the DHR, DIST, and Innovation colleagues that the organization can learn from this journey, and perhaps be inspired to find new ways to advance the systems that power UNHCR’s mandate.