1 year of chatbot development — 10 best practices
At the Finnish Immigration Service (Migri), we have been developing our chatbot for a bit over a year now. It has been live since May 2018 and has since had more than 45.000 conversations with its users. Kamu gets good feedback from the immigrants, and it has become an integral part of Migri’s customer service. On top of that, we are expanding the content design team soon. We think, it is a good time to look back and answer some questions: What have we learned in one year of designing content for Kamu? What are the best practices that are not only useful for us at Migri, but that other organisations could adapt?
Here is our list:
01 — Decide on the content area to cover
With no doubt, the most important best practice is: Decide on your content area based on your users’ needs. It is not about what your organisation thinks customers need, but the users’ real situations. We usually prioritise content based on our substance units’ and customer service experts’ opinions, as well as organisation’s statistics. Since we follow conversation logs, we also use frequently asked questions for prioritisation.
As a second step, you need to understand what effort goes into the content area you have chosen. In other words: how wide is the area? Can you break it into smaller pieces that can go live independently of each other? Aim to get content out quickly, but make sure that each chunk of content you publish brings value to your customers.
02 — Put yourself in the shoes of your user
When creating content for a chatbot, it is equally important to understand your users. We try to put ourselves in the shoes of the immigrants: What are their needs? How would they ask this question? Is this the answer they would expect? If we are not sure, we try to consult outsiders. Those are often people in the organisation who do not work in this particular substance area.
Understand how to talk to your users in the chatbot. The language knowledge within our user group, immigrants, varies a lot — from elementary level all the way to native speakers.. The magic lies in using simple language but not simplifying too much. To do this, consult people who talk or write to your users on regular basis and talk to the users yourselves! Do user testing! Often! Everyone in the team! It will help you gain a deeper understanding of the customers’ use of the chatbot content and their frustrations with the bot.
03 — Think about the personality of your chatbot
Establishing a personality for your chatbot helps your content creators when writing the answers. The personality may include the main traits of your bot: Is it proactive or reactive? What’s its name and gender? How does it address people?
We recommend testing a few different personalities with your users — not only because it is great fun, but also because it is the best way of learning what customers expect from your organisation and your chatbot long before you launch the pilot.
04 — Establish common content creation guidelines
Establishing common content creation guidelines early on in the process will have a long-term benefit and impact. They ensure consistency of the produced answers when several content creators are involved. Even if you start with a small team, over time it will grow, or one content designer replaces the first one. In the content creation guidelines, we considered our target group, our organisation, our chatbot’s personality, as well as the technical restrictions of our chatbot software. For example, think about the spelling logic to use in English (British? American? Canadian? Australian?) or how to format numbers (1000 or 1.000 or 1,000?). It will confuse users if you are inconsistent with these and take a long time to change later.
05 — Train your content creators in writing
In our organisation, we do not have content creators who are trained writers. Our content creators are motivated and learn a lot by doing, giving each other feedback and hearing feedback on their texts from users. Nevertheless, consider sending your content creators to specific trainings on content creation in digital channels and writing for your target group. We have used so-called “Plain Finnish language” trainings for our content creators and find them extremely valuable.
06 — Understand your substance
We recruited all our current content creators from within the organisation. This means that all of them are substance experts in different areas. This is a big benefit when creating content for our chatbot. But even if you hire substance specialists, prepare to spend time with other substance experts who specialise in the area you are working on. Learn from your substance experts and listen to their opinions. We use everything from brainstorming to workshopping and review groups to get feedback.
07 — Well-planned is half done
We have started to teach chatbot content creation to other public organisations in Finland. What we see often is people rushing into the chatbot software. But to us, a big factor is planning the content well BEFORE putting it into the software itself. This means that we draw the conversation logic on big sheets of paper. We use post-its to imitate conversation flows. Sometimes we even play through an example conversation to understand how it feels. Putting time to planning the content helps us in taking user orientation seriously.
08 — Ensure teamwork among your content creators
When multiple people write replies for one single chatbot, they need to ensure consistency. To reach this, we brainstorm together. We discuss drafts of the content in early stages. We proofread and translate each other’s content. Content creation guidelines serve as a basis, but practical collaboration between the different writers is essential.
09 — Get feedback on your content from substance experts
You might have guessed this one already after reading number 6, but it is so important to incorporate feedback from the specialists that we mention it again here. Before your content goes live, talk to your substance experts. We have a review group consisting not only of experts from all decision-making units, but also legal, communications and customer-service professionals. All this is to ensure that the content is as accurate and as good as it can be. Involving this many people takes time away from their other work. This is why it is important to ensure that the substance experts have dedicated time to take part in content development. We produce the content in one language first. Substance units approve it, and then we translate the content into the other languages that our chatbot serves the immigrants in.
10 — Study your logs regularly
If you have followed all of our best practices above, you have done everything to ensure the best possible quality of your content before it goes live. Still, the best way to see what works and what doesn’t is to follow the conversations that real users have in real situations. This is why you have to follow the logs (and make sure you invest in a chatbot software that lets you do this). You may find out users use different words than you expected or that they expect a different answer. When you spot errors, you can make changes immediately. This will happen all the time. The more users you have, the more errors you will find. Make sure to prioritise what you fix immediately and what you put into the backlog of your content production team for the future.
Make sure you have dedicated time and responsibility within the team to follow conversations. This is also a great way for the content designers to learn about the kind of language users use. Additionally, it lets you discover new content areas which users are expecting answers on, but you are not covering yet.
These are our ten best practices when creating content for chatbots. They were created with input from our content designers Laura Halonen and Katja Rintala as well as our AI supervisor Mari Humalajoki. We share our thoughts as a reflection for ourselves, but also to generate discussion and hear what others find as useful tools and methods when creating chatbot content.
Looking back at one year of creating content for our chatbot Kamu, the one thing that we have not dedicated enough time to is user testing. This is why it doesn’t appear as one of the ten items in our list. We were so busy getting Kamu up and running, extending its content and thinking about new Kamu experiments that we didn’t have enough resources in the team to take user testing serious enough. And we clearly want to change this in 2019!
Editors: Laura Halonen & Mari Humalajoki
