Can design reduce biased behaviour? Einstein said “It is harder to crack prejudice than an atom.” But back then, they didn’t have UX design.
The Problem with Prejudice
“French bakeries make the best croissants”, I recently told a colleague. Where’s the nearest French patisserie?”
With that judgmental statement, I had immediately ruled out the large number of Australian bakeries that have mastered the art of croissant-baking.
And that is one of the problems with prejudice.
We accept that what we feel is right, without considering alternative options. We limit our perspective, options and experiences. And when we do this, we unfairly dismiss people, situations and opportunities.
Most of the time we are unaware that we are doing it. And most of the time, we fail to recognise it in others.
Introductions
Before I jump into the case study, here is a bit of background.
I’m Bella, a UX designer at SEEK. My team helps hirers (who post job ads on our platform), to quickly identify suitable applicants. We auto-suggest bespoke screening questions that hirers can include in their job ad.
When candidates apply for the role, they answer the screening questions and we rank them according to their answers.
One of the most popular questions that hirers use is: ‘Do you have the right to work in Australia?’ This question helps them quickly assess an applicant’s working rights in Australia.
Depending on the company’s requirements, this question is an easy way to identify suitable candidates.
So what’s the problem?
There are only a few industries where you need to be an Australian citizen to be eligible for work e.g. government and defence jobs. Only 2.5% of all job ads posted on SEEK fall into this category. These roles have a legitimate reason for employing citizens — typically they need a security clearance.
Yet 18% of hirers on SEEK select the “citizen” option as their only preferred answer. Even when they aren’t recruiting for government or defence sectors.
Further analysis shows that 20% of suitable candidates (who meet all the required criteria) are not citizens. This is a huge market that our hirers are potentially missing out on!
My team, pretty much consists of permanent residents (not citizens). So this situation struck a personal chord. Using the stats above, only one in five of us non-citizens would be given a fair chance at a job.
We were concerned that:
- relevant and suitable candidates were being overlooked and
- both candidates and hirers were missing out on connecting with each other.
Therefore we wanted to reduce the number of hirers that selected the “citizen” option (shown below).
We currently draw attention to the impact of selecting the “citizen” option with a message. It was displayed if hirers selected only the “citizen” option. If hirers included permanent working rights in their preferred answers, the message disappeared.
First experiment: adapting the message
To decrease the percentage of hirers selecting the “citizen” option as the only preferred answer, we reviewed our design.
We suspected that:
- the content of the message was too vague. It didn’t communicate the real impact of selecting “citizen” as the only option.
- the position of the message was poor — it should be closer to the action.
- as well as being out of sight at the bottom of the section, the design was not eye-catching enough.
So we changed the message and updated the design:
We ran an A/B test between both versions for 4 weeks with 10,000 hirers. The team was confident that Version B would be the clear winner. It was a no-brainer. The message was impossible to miss. And why would hirers want to exclude 20% of suitable applicants on purpose?
At the end of the test, the results were conclusive — both versions performed exactly the same.
We had not improved the situation. At. All.
The experience did reinforce a few valuable lessons though:
- don’t assume you know your users
- it’s okay to admit that you were wrong
- it’s totally fine to eat humble pie if you focus on learning from your experiences.
Second experiment: learning from our customers
We interviewed hirers to learn more. We selected hirers that weren’t recruiting for government/defence jobs, but still selected “citizens” as their only preferred answer.
They agreed with us on all points:
- The new copy made a compelling argument to select more options in the preferred answers. They liked that we were being transparent and communicating the percentage of applicants that they were missing.
- Yes, the purple design was eye-catching and stood out more than the previous message.
- And yes, the positioning of the message was better and made it easier to see.
So why were they still only selecting the “citizen” option?
There were two reasons:
- Some hirers didn’t understand the difference between the “citizen” and “permanent” options.
- The majority of hirers admitted they only want to recruit Australian citizens. They thought Australian citizens were more suitable to roles than foreigners with unrestricted working rights.
Our challenge was to make the visa options more clear and reduce the bias.
The first part is easy, the second part, as Einstein stated, is a tougher nut to crack.
Iterating on the design based on our learnings
We set out to change the UX design of the “right to work” question, and this time we presented the options more aggressively:
- We decreased the number of visa options from 5 to 4.
- We collapsed the citizen and permanent option into a single “permanent unrestricted” option. Hirers that selected this option would automatically include citizens and permanent residents as suitable applicants.
- We introduced a link to an “advanced options” view. In the advanced options we list all possible visa types. This was specifically aimed at the 2.5% of ads that need applicants to be Australian citizens.
- We removed all messaging, as it was no longer needed.
Then, we ran an additional A/B test where we tested Version A (original design) against Version C (latest design).
The Results? No more prejudice!
After 3 weeks we reached statistical significance:
We have now rolled Version C out to 100% of all hirers. The proportion of hirers selecting the “citizen only” option has since increased to 2%. Which makes sense, as we previously learnt that 2.5% of job ads require applicants to be Australian citizens.
Did we manage to change hirers’ biased preconceptions? Probably not.
But we did manage to change their behaviour.
And by doing that, most importantly, we ensured that all applicants are given a fair and equal chance at a role.
Their suitability for the role will be evaluated based on their skills and experience and not determined by their place of birth.
This group of immigrants is extremely pleased with the results.
Written by Bella Schoeppe
Senior UX Designer at SEEK
