Large Language Models for Multilingual Artificial Intelligence Assistants

Kore.ai
4 min readApr 21, 2023

--

The Kore.ai XO Platform is truly global and multi-lingual. When customers interact with help and support intelligent virtual assistants (IVA’s) they prefer to interact in their native language or the one language that is most comfortable for them. In that way, they can express their queries precisely.

When your business interacts with customers around the world a multilingual intelligent virtual assistant helps you ensure that everyone receives the same level of quality service, irrespective of the language they speak. The Kore Platform XO artificial-intelligence-powered platform also provides a distinctive feature by adding multiple languages to your virtual assistant and training it by using your own language as well as the language you are programming the bot with.

Out of the box the Kore XO Platform supports about 120 languages, with more coming soon. Languages supported by the platform range from Arabic to English and even Swahili. It is easy to manage any combination of languages that you wish.

How XO Platform Supports Multiple Languages for Customer Support

So there are three different ways by which Kore supports language:

  1. One is with a multi-language model that has been built into the Kore AI XO platform.
  2. The other one is through a curated model. You actually do the translations manually, so you have the most accurate responses coming back from the bot.
  3. The third one is through a translation model.

So using the combination of those three, and using LLM’s through Design Time, Kore can support most languages.

How Large Language Models Support Multiple Languages

The whole concept of Large Language Models is based on the ability for these models to understand accurately what a person may be intending to say. So the Natural Language Processing (NLP) part of it is highly improved with the amount of training data that has been used to build these models, as well as the ability to generate the appropriate answers to questions.

You can start building the intelligent virtual assistant (IVA) with one language and enable additional languages when needed. As the customer chats with the chatbot in his or her preferred language, but then changes to another language (say from German to English), the intelligent virtual assistant recognizes this change and switches to English.

A multilingual IVA is an AI-powered intelligent virtual assistant that can converse with customers in multiple languages instead of defaulting to English. The Kore.ai XO Platform supports building your intelligent virtual assistant in multiple languages. The default language selected when creating an assistant cannot be changed, as all of the bot configurations and Natural Language Processing (NLP) training are processed in the selected default language. For instance, you can program the VA using English commands, but train the bot to converse in the conversation language of German.

As the model is language agnostic, you can train the model in any of your preferred languages or a combination of languages.

The Kore.ai XO Platform can help you to:

  • Configure the required language
  • Identify the NLU Model
  • Identify various language definition modes — Basic Mode, Advanced Mode, Use Language Pack
  • Manage VA and User responses translations
  • Manage Multilingual NLU model
  • Train and talk to the virtual assistant

There are six steps involved in training your intelligent virtual assistant to recognize and converse fluently in multiple languages:

  1. Configure required language
  2. Identify NLU Model
  3. Identify various language definition modes — Basic Mode, Advanced Mode, Use Language Pack
  4. Manage VA and User responses translations
  5. Manage Multilingual NLU model
  6. Train and talk to the virtual assistant

Advanced mode allows you to copy the definitions of the Dialog prompts and messages, knowledge graph, small talk, and training data.

The NLU Language is the one that you train the assistant with to identify the user’s intents. The NLU model is built using the NLU Language that you choose. This language can be the same as the Conversation language or any other supported language.
You can choose a combination of Conversation Language and NLU Language if required. For example, you want to enable the German language for your assistant but train using English. You can also take advantage of translation services within the platform.

As the customer chats with the intelligent virtual assistant in his or her preferred language, but then changes to another language (say from German to English), the VA recognizes this change and switches to English.

As you look forward from a business point-of-view if customers want to, or enterprises want to transform the experiences that they put out to their customers, or to their employees, or other types of partners, you know those experiences involve executing an entire dialog whether just asking a question, producing some information through an enquiry, or executing a transaction.

The Kore.ai XO Platform truly democratizes the process of creating multilingual intelligent virtual assistants. No additional skills are needed to do this other than the skills already learned in creating English VA’s. It is recommended that the person doing this work is a Product Manager with familiarity in the industry the VA is for, such as banking or pizza restaurants.

The Cost of Creating Multilingual AI Assistants

As for the cost of creating multilingual AI assistants by licensing the Kore.ai XO Platform, it depends on whether the customer directly accesses an Open AI LLM, or uses the LLM’s that the Kore.ai XO Platform uses. Often those are from Open AI. It is the customer’s choice.

Kore.ai will be adding other languages in coming weeks. They have a roadmap of rolling it out for each of the LLM’s. Using the Few Shot and Zero Shot methods Kore.ai is taking it language by language because Kore is validating which LLM is better in terms of Open Source LLM’s, and fine-tuning them for language.

Learn more about the Kore.ai XO Platform.

--

--