Beyond the ChatGPT hype: Conversational AI and diversity, equity and inclusion

Helen Babb Delia, development partner at FORWARD — The RMIT Centre for Future Skills and Workforce Transformation — and director Peter Thomas, writing in collaboration with RMIT FORWARD with development partners Pete Cohen, Inder Singh, Kate Spencer, Sally McNamara, Daniel Bluzer-Fry, Soolin Barclay and Courtney Guilliatt.

We recently ran a workshop for Conversation Design Institute (CDI) Foundation’s ‘Conversation Design Festival: Making AI Speak Human’. Across the one-day festival, we joined with 1000 participants and 10,000 viewers from over 25 countries.

Our workshop, ‘Creating a Diverse, Equitable and Inclusive Conversational AI Industry’, focused on the reality, challenges and opportunities for inclusion in conversational AI and the future of work.

The conversation we need to have

Our daily interactions with brands — from customer support to conversational commerce — are increasingly happening via intelligent assistants. The industry that creates these experiences — this industry — is becoming more sophisticated and is scaling to meet the needs of potentially billions of people.

As this happens, we think it’s essential to place concepts of diversity, equity and inclusivity centre stage.

We’ve learned in many domains that a company, a workplace or an industry that embraces difference and where diverse voices are heard, respected and valued will be more successful for people, for the organisations in which they work and for society as a whole.

Of course, as generative AI becomes more widely used, some of these issues are being foregrounded.

For example, ChatGPT is facing and will increasingly face issues of invisible bias and diversity of knowledge. Only in the last few days, we’ve seen how far humans are in the loop of what ChapGPT does and come to learn about some of the issues surrounding the ethics of human-based content moderation. Or, take image tools like DALL.E that respond to prompts based on the datasets it was trained on. These datasets feature a much higher percentage of white males, and initially, this was reflected in the pictures that DALL.E generated.

Of course, much overt undesired content — that is sexual, hateful, violent, or promotes self-harm — is addressed by OpenAI’s moderation endpoint and other tools. But, one might argue, this is just making it seem less biased.

Much of the data used to train large language models, in the form of the common crawl dataset, has been created by white males in western countries. Those same people have had the largest role in creating the technology. The potential exists for generative AI to widen the inequality gap and reinforce many of the biases that are the undercurrents in the internet and in society more broadly.

There are relatively few Black people and other people of colour working in AI. At some of the top tech companies the numbers are bleak: Black workers are only 2.5% of Google’s workforce and 4% of Facebook’s and Microsoft’s. It's the same for gender globally, only 22% of AI professionals are female, while 78% are male.

For the conversational AI industry, while the numbers may be more encouraging because of the diverse talent pool the industry draws on there’s little concrete data. But regardless, issues about diversity, equity and inclusion are important in this industry as any other.

We have long known that gender, ethnicity and other types of diversity affect the bottom line. Companies that lag in gender and ethnic diversity among their workforces, management teams, executives, and boardrooms are less likely to be successful. If an organisation’s leadership and workforce do not reflect the diverse range of customers it serves, its outputs will be less acceptable to those customers.

The last thing that technology that assists and replicates human interactions should do is uphold old human biases and perpetuate harmful and inaccurate stereotypes. Instead, it ought to be the best that humanity has to offer, rather than the worst.

Yet even though companies say that they’ve tried to address issues of diversity, equity and inclusion, the needle has barely moved. Since 2014, when the large tech companies began publishing annual diversity reports, few have made much ground in terms of ethnic diversity. Some have made small gains in gender diversity.

This lack of progress shows it’s not a challenge that can be solved easily. In our workshop at the Conversation Design Festival we wanted to start the conversation about diversity, equity and inclusion and bring these issues to the top of mind so that when people encounter them, they may be more prepared.

Challenges and opportunities

In our previous writing, we have outlined the fundamentals of diversity, equity and inclusion and in our workshop, we provided a very brief overview of definitions and terms to set the scene.

For this workshop, we wanted to draw on the collective wisdom of people with experience in the AI industry and what they’ve experienced or witnessed. We asked for their views on challenges and opportunities, which you can see on the Miro board we ran live during the workshop with over 100 participants.

The challenges that our participants identified fell into three broad areas:

• Challenges in the industry around attraction, retention and creating an inclusive conversational AI industry. This includes attracting people who don’t know where to look for jobs and what is expected of them, or ‘women in tech’ being too narrow a focus that doesn’t consider other intersections of race, gender or other identities.
• Challenges around data, models and other technology used in conversational AI. This includes safeguarding issues, models needing to be assessed by a bigger group of people and the dubious quality of free or scraped data.
• Challenges around product ideation, testing and rollout. This includes a lack of diversity in user testing groups, a lack of time for ideation and the need for more cross-functional approaches across organisations.

There was a lot of discussion about the opportunities for the conversational AI industry and products to promote inclusion, including:

• Highlighting key transferable skills to attract people from ‘outside’ the industry and showing examples of people from diverse backgrounds who are succeeding.
• Investing in data sets, tools and skills and experience in working with inclusive datasets in the workforce.
• Bringing a diverse group of users in for co-creation rather than waiting until there is a product to test. Investing in ways that conversational AI can detect and reduce human bias or make information more accessible to people (such as information available via WhatsApp in a way that’s understandable for the target audience).
• More inclusive practices in conversational copywriting and/or hiring storytellers, writers, people with psychology majors, language experts and most importantly, people with compassion and acceptance
• Demystifying this new technology and being clear on the opportunities and value that conversational AI can bring

We invite you to reflect, comment on and add to what we and the workshop attendees identified. This hopefully will help drive progress in the areas of diversity, equity and inclusion and improve the conversational AI industry.

We are grateful for the contributions that all of our attendees made and to the organisations and people that ran workshops during the event, including American Red Cross, Rasa, Talent Data Labs, JPMorganChase, Intuit, Voiceflow, Microsoft, Humanfirst and UltimateAI.

FORWARD is the RMIT Centre for Future Skills and Workforce Transformation.

Our role is to build an innovative learning ecosystem at scale, create new collaborative applied research and invent next-generation skills solutions that will catalyse workforce development in the future-oriented industries crucial to Victoria’s economic renewal.

We lead collaborative applied research on future skills and workforce transformation from within RMIT’s College of Vocational Education, building and scaling the evidence and practice base to support Victorian workforce planning and delivery and acting as a test lab for future skills to develop and pilot new approaches to skills training and education through digital transformation and pedagogical innovation.

We leverage RMIT’s multi-sector advantage to translate research insights into identifying workforce requirements and the co-design of practice-based approaches with industry.