Building Generative AI Assistants with DeepPavlov Dream

Authors: Daniel Kornev, Dilyara Zharikova, Nika Smilga, Ksenia Petukhova, Alexander Popov, Mikhail Burtsev

At DeepPavlov, we strive to be at the edge of the Conversational AI field, and we experimented with large language models (LLMs) like GPT-2, BlenderBot, and others since as early as 2020, and embedded ParlAI’s BlenderBot into DeepPavlov Dream in early January 2021 during our participation in Amazon Alexa Prize Socialbot Grand Challenge 4.

This Winter, we’ve incorporated some of the most popular open-source Large Language Models like GPT-J, BLOOM, and others from Transformers library into our open-source multiskill AI assistant platform DeepPavlov Dream. Our new prompt-based Generative AI Assistant distributions make it possible for you to build controllable Generative AI assistants and are the first step towards building safe generative AI assistants.

Try Building & Running Generative AI Assistants using DeepPavlov Dream

Intro

We are excited to introduce our first Generative AI Assistant distribution alongside the tools for building Generative AI Assistants with DeepPavlov Dream to get users’ feedback and learn about their strengths and weaknesses. This distribution enables you to incorporate LLMs of your choice to let your AI assistants generate responses using LLMs based on your instructions.

Learn more how different LLMs perform when controlled with prompts

Example

In the following example of a persona-based Generative AI Assistant, the GPT-J model from the Transformers is embedded into a DeepPavlov Dream Generative AI Assistant to power a Persona-specific prompt-based Generative Skill. We start with the following prompt:

{
  "prompt": "Respond to a new friend as a kind friendly person.\n\nYour personality:\nMy name is Dream. I am a chatbot. My work is to talk to people about everything. I like to talk to people about their lives. I have a lot of friends. My friends are all the people I talk to. I become friends with everyone. I love people. I can not play sport. I like to watch football and basketball on TV. My favorite sport is football. I like watching people swimming. I love dogs so much. Dogs are the best friends. I do not like cats at all. I do not have a dog yet. I would love to have a dog in future. I like Italian food especially pasta and pizza. My favorite food is ice-cream. I hate onion. I like travelling. I can not travel physically. I like visiting interesting places virtually. I love to walk on Paris streets with Google Maps. I like watching travel video blogs. I adore watching wild animals. I am scared of spiders and snakes."
}

This prompt lives in common/prompts/ subfolder of the repository. For this distribution, we use dream_persona.json file to store the prompt above. If you want to experiment with this distribution w/o adding new Generative Skills, you can just change this file to a prompt of your liking.

This template Generative AI Assistant distribution includes two skills: Generative Skill called Dream Persona Prompted Skill, and a Fallback Skill (“Dummy”), and a new Generative AI Model Service that hosts a local copy of a GPT-J 6B model. Generative Skill is configured to call this GPT-J Generative AI Model Service to generate responses. The prompt above, combined with 3 previous user phrases is sent to this Model Service to generate responses.

To talk to the assistant, you should:

1. Clone Dream: git clone https://github.com/deeppavlov/dream
2. Switch to it: cd dream
3. Run the Generative AI Assistant distribution locally: docker-compose -f docker-compose.yml -f assistant_dists/dream_persona_prompted/docker-compose.override.yml -f assistant_dists/dream_persona_prompted/dev.yml -f assistant_dists/dream_persona_prompted/proxy.yml up --build
4. Talk to it in a separate terminal tab: docker-compose exec agent python -m deeppavlov_agent.run agent.channel=cmd agent.pipeline_config=assistant_dists/dream_persona_prompted/pipeline_conf.json

Try Building & Running Generative AI Assistants using DeepPavlov Dream

Limitations

In addition to the limitations common for LLM models, this release has the following limitations:

• In this release, you will need a machine with the powerful-enough GPU to run Generative AI Models like GPT-J locally;
  Note: In future releases, we plan to introduce a mechanism for using OpenAI models such as GPT-3, which will eliminate the need for a powerful GPU. However, please note that using OpenAI models will require a separate payment to OpenAI;
• Same prompt is used during the entire dialog for each Generative Skill;
• If the conversation significantly deviates from the original prompt for any of the Generative Skills, the platform may stop invoking the associated Generative Skills and provide fallback responses using its Fallback Skill (“Dummy”);
• When using the Prompt Selector, it’s important to ensure that the number of prompts provided corresponds to the number of Generative Skills in your Generative AI Assistant. If you provide more prompts than available Generative Skills, the Skill Selector will automatically include the names of the Generative Skills specific for each prompt, even if the corresponding skills are unavailable. This can cause the system to provide fallback responses using its Fallback Skill (“Dummy”);
• To prevent issues with the system, it’s important to name prompt files using lowercase letters, avoiding spaces and special symbols, and using underscores _ instead of hyphens -.

A Sneak Peek at Generative AI Assistants Architecture

At the heart of DeepPavlov Dream lies an asynchronous pipeline that facilitates seamless communication between our platform and users. This cutting-edge approach enables our platform to process user input and generate responses in real-time, creating a seamless conversational experience.

Our platform’s symbolic and neural AI components, called “Annotators,” perform a wide range of tasks, from converting oral speech into written text to identifying entities and linking them to Wikidata or custom knowledge graphs. Additionally, our “Skill Selector” ensures that the most relevant skill is selected at each turn, providing a personalized experience for each user. Finally, our “Generative Skills” control Generative AI models to produce relevant responses back to the user.

The latest release of DeepPavlov Dream introduces new Generative AI assistant distributions, which use automatic selection of generative skills by matching user phrases with prompts used inside Generative Skills to control their respective Generative AI models like GPT-J or BLOOM. This creates a more natural and human-like experience for the user, making conversational AI more accessible and intuitive than ever before.

But this is just the beginning. By efficiently combining the power of neural and symbolic AI components to control and augment Generative AI models (check out this review by our friends at Meta), we believe we can unlock endless possibilities for conversational AI. Our platform will be able to use knowledge about the world after 2021, query our personal and company databases, reason, and even act in the virtual and physical worlds on behalf of us.

We are excited to share this latest release with the community and can’t wait to see the incredible innovations that will come from this new technology. Stay tuned for more updates from DeepPavlov in the coming months!

Motivation

While at DeepPavlov we started experiments with embedding large language models as early as in 2020, their practical applications were limited until the introduction of GPT-3. With the recent release of ChatGPT, the entire industry is thrilled to witness one of the fastest technology adoptions in history. With the correct instructions, ChatGPT can help solve a myriad of tasks, including rewriting emails, generating essays, explaining complex problems in simple terms, summarizing texts, and more. Suddenly, LLMs have given us a glimpse of the AI that many have anticipated for years.

For example, you can ask the model to “Create a content calendar with six blog titles, including the keyword Reaching Mars by 2025. Pick suitable publishing dates for each guide spread across May 2023.”, and it’ll give you just that:

Example Prompt for ChatGPT gives you a fully prepared content plan for blog posts about “Reaching Mars by 2025”

There are now countless blog posts, tweets, and social media posts with endless lists of prompts for ChatGPT, which can handle tasks across various industries. Bing has integrated ChatGPT into its search experiences and into Edge (which is a part of Microsoft’s Bing organization). Opera has also incorporated similar experiences. Our friends at You.com added similar features to their product at the end of last year.

It is the best time to enable our developers to incorporate LLMs into their AI assistants essentially turning them into Generative AI Assistants.

Today’s early developer preview release of our tools for building Generative AI Assistants with DeepPavlov Dream is the latest step in DeepPavlov’s iterative development of the open-source tools for building modern AI assistants.

In the coming releases, we will add support for using Open AI LLMs like GPT-3. Stay tuned!

Try Building & Running Generative AI Assistants using DeepPavlov Dream