CivTech Week #3: We’ll plant 100 trees in return for 15 minutes of your lovely Scottish accent

In this autumn of shortages, can we add our shortage to the list? We are currently experiencing a shortage of Scottish accents, a vital raw ingredient for testing and training a voice AI system to be deployed in Scotland.

If you’re Scottish and have a spare 15 minutes to talk to us (and our bot), we’d love to hear from you. Contact michael@sidelabs.org. We’ll plant 100 trees to say thank you for time.

As a quick recap, we started last week by evaluating our user research into our voice AI system for Citizen’s Advice Scotland. We realised:

• Scottish accents are tricky for AI. In particular ‘A’ and ‘E’ are hard to tell apart. On the other hand, our AI beats us at telling a Scottish ‘J’ from a Scottish ‘G’.
• Humans don’t mind talking to a robot voice, as long as they feel they are making progress and getting directed somewhere.
• The default ending to the call script ‘Have a wonderful day’ strikes the wrong note. If you are speaking to Citizens Advice, presumably you aren’t having a wonderful day. The voice AI can’t be too chipper.
• People like the robot voice to be formal! We got feedback that starting with ‘Hi’ is too informal, and ‘Hello’ feels about right.
• Voice AI works considerably better on standard phone settings rather than on speaker phone. But for some reason, it seems more natural to us to call a helpline on speaker phone. (Is that just us?)
• We may need an extra question in the dialogue flow. Currently we just ask the caller for their postcode. To improve the routing, we may also need to ask what subject the caller wants to ask about or their local council details

We offered some of the feedback from user testing to one of the potential AI software providers. They suggested we find native speakers to train the bot to better catch postcodes. Since then, we’ve been busy arranging for volunteers to help optimise the AI. Now we are looking to do a retest, with the more fully trained systems.

In other news, we also researched the logic behind matching a postcode to a bureaux. We spoke with Basestation, the Edinburgh agency that built the Citizens Advice Scotland’s bureau finder. This conversation was super helpful.

This finder doesn’t just point to the nearest bureau, it has to check if the bureau is in the same local authority area as the caller. Beyond this, there’s further room for improvement. For some islands or city centres, extra logic is required to make the correct recommendation.

We’re still scratching our heads slightly on how best to generate accurate recommendations that incorporate all these edge cases of local geography.

Stay tuned to hear how we get on!