What We Have Learned, Building and Testing a Youth-Powered Digital “Sexual Reproductive Health Bot”
The PSI Team is designing and testing a Sexual and Reproductive Health (SRH) chatbot with young people in Kenya as a tool (vehicle?) for generating insights into how pleasure content influences engagement with the bot itself and outcomes. With investment from UK Aid, we have completed the second sprint of the project with the aim of developing a high-fidelity prototype aligned to our target audience User Experience (UX) needs with a focus on technology functionality.
The bot, in brief
The bot prototype is running via WhatsApp, Facebook messenger, and a microsite. It has an embedded facility locator and self-referral capability. The prototype allows users to explore content related to modern contraceptive methods, menstrual health, and provide feedback on content relevance and get clinic location support services.
How we approached our research
Our team brought together online focus groups to finetune this SRH bot to the distinct needs of Kenyan young people ages 18 to 29. To test the prototype, 10 participants interacted with the bot during their free time using a pre-designated channel over a period of 72 hours, While users’ first interaction with the bot was reflective of their specific priorities, they were encouraged to explore as many “branches” of content as they could to test the functionality of the bot.
Below, learn about some of the insights we’ve collected as we dig into what it takes to design and develop an SRH bot, with Kenyan youth, for Kenyan youth — during COVID-19 and beyond.
Testing the bot: 5 dos and don’ts
- Participants need to be introduced to the SRH bot — and consent to using it — in a way that’s captivating, relatable, and straightforward.
While some participants didn’t mind our introduction to the SRH bot (see below), a majority of them felt that the introduction wasn’t captivating enough and weren’t motivated to move forward.
What’s more, males opted out immediately because they felt that the welcome message positioned the bot as a female product. Male participants viewed contraception as a “female affair” and weren’t motivated to support their partners in the decision-making process around family planning. Most male participants described themselves as being in “situationships,” meaning they weren’t committed to just one partner and left decision-making around contraception to their female sex partners.
Obtaining consent also presented an issue. The SRH bot asked for consent by posing the question, “do you agree to continue?” after giving a short introduction to the service. We found, for our participants, that this question did not equate to consent for data use and follow-up communication.
2. Participants are happy to share their data — explicitly sharing how and where users’ data will be collected, and how it will be used is key!
Participants were comfortable sharing their gender and age. In fact, using this data to categorize them into age groups made them feel safe and unjudged when interacting with the SRH bot. Asking for demographic information also made participants take the bot more seriously, as they are asked for the same information for government planning and reporting purposes.
However, participants were not aware that their phone numbers (on WhatsApp) and Facebook IDs (on messenger) were being auto collected for future engagement needs and creating referrals. This was stated in the privacy policy but many participants didn’t read this because they wanted to save data. We know now: we must explicitly share with young people how and where their data will be collected, and what it will be used for. PSI’s Design Ethics for Youth-Centered Innovation offers a strong reference point.
3. Content needs to be brief, exciting, and most of all, relevant.
A common complaint from participants was that the content was too serious. As a 24-year-old female participant in the focus group said, “I like [the SRH bot], it is easy to use, however, the content is too serious, please spice it up!”
While users found the bot very responsive and defined it as (mostly) easy to use, they also told us that the content was too plain. They suggested spicing it up with GIFs, emojis, photos, illustrations, more color, and links to videos.
And while the content was short and to the point, the users reported the information provided as not new, exciting, or even relevant, contrary to insights we have from qualitative research with young people in many other studies that suggest young people want straightforward, fact-based information about FP given the swirl of misinformation they generally encounter. In addition, users didn’t correctly interpret some of the titles and subtitles on the menus provided, which meant they were guided to the wrong content and that they reported gaps in what information was available to them via the bot. This meant that users responded to the bot with questions rather than simple answers (see right), indicating a shift in preference from a structured bot to an interactive AI driven bot or link to a direct chat with a cyber educator.
Users also questioned the accuracy of the information provided. They reported grammatical errors in the questions and responses from the SRH bot, which made it feel “cheap” and less legitimate. One piece of new information provided by the bot also called its accuracy into question: the bot reports to users that emergency contraception is efficient within five days of unprotected sex, but participants had heard that it was only effective within three days. Although five days is accurate, because it didn’t match current knowledge, it led participants to doubt the accuracy of the information provided.
4. Be aware of the limitations of each channel and how they affect the user experience.
WhatsApp was the channel of choice for most of our participants due to its ease of access, frequency of use as a communication channel, and good security. However, it has its limitations when it comes to online focus groups. On WhatsApp, repeat bot users had to log their demographic information each time they used the bot. The platform does not currently have the ability to record to recall user demographic data. Additionally, links are not easily accessible on WhatsApp — while they appear and are easy to click through on Facebook Messenger, this is not the case for WhatsApp.
Facebook also had its limitations. Some users of Facebook messenger use the “lite” version of the app, which uses less data, making it attractive to users, but does not allow the use of rich features — such as buttons and quick replies — that are necessary for use with the SRH bot.
5. The referral concept in the app needed to be explained better.
Some users found that the concept of referral to a healthcare provider was not well-explained. One 27-year-old female participant thought that the bot automatically made an appointment for her with a provider: “Oh my God… what just happened? Did they just make a booking to a hospital? How did this message end up on my phone? Gosh, so I’m I expected to go to the hospital? I just clicked it out of curiosity? What does this mean?”
So what comes next?
These insights led to an integrated version bot launched at the end of August. We have kicked off Sprint 3 with traditional SRH content in English to generate baseline data and will include pleasure content in the last week of September. The referral concept and how it works should be highlighted for users from the beginning so that they can effectively use the SRH bot and redeem the referral if wanted.
Want to learn more? Drop us a note: coduor@Psi.org
