Introducing a framework designed to effectively manage AI-generated data within production systems.

By Dall.E Model

In the realm of artificial intelligence, GPT (Generative Pretrained Transformer) models have emerged as a powerful tool for a variety of tasks. Some are now well known by the general public, as text generation or question answering, but many other use-cases are now being discovered. Perhaps, in the near future, we’ll seing generative models being used to detect sensitive content in corporate emails or witnessing fully functional customer support departments operated by AI.

However, to fully take advantage of the benefits of these models in production systems, it is essential to develop techniques that effectively integrate Natural Language (utilized in generative models) with the required Programming Languages.

The following concerns need to be addressed to solve this problem:

• How can I incorporate the data I possess (such as user inputs, database data, etc.) into prompts written in natural language, enabling them to read or manipulate this data?
• How can I structure the response from GPT model (which is in Natural Language) and convert it into objective data that i can programatically manipulate?

I am documenting here some techniques that I use in my own projects. Despite using the Python language in this article to save text space, they can be applied in any programming language.

This article doesn’t aim to address the initial steps of using ChatGPT API. For that, you can refer to this excellent article from DevGenius. Instead, we’ll concentrate on crafting prompts that produce responses ready for programmatic consumption, as well as on how to interact with AI-generated knowledge.

Available Endpoints

OpenAI has several language models available:

1. GPT-3: This is the third iteration of the Generative Pretrained Transformer models. It is a powerful language model that can generate human-like text. It’s used for a variety of tasks, including translation, question-answering, and text generation.
2. GPT-4: The latest and most advanced model from OpenAI. It is capable of solving complex problems with high accuracy and has enhanced creative and collaborative capabilities. It is currently available via a waitlist. (May of 2023)
3. Davinci: This model is highly advanced and capable of understanding complex instructions and generating detailed and nuanced responses.
4. Curie: This model is a balance between cost and capability. It’s useful for tasks that require a moderate level of understanding and creativity.
5. Babbage: This model is the most cost-effective option for tasks that require less creativity and more straightforward answers.
6. Ada: This is the smallest and fastest model. It’s ideal for tasks that require quick, simple responses.

In my experience, the framework defined in this article was functional for GPT3 and Davinci models. It is expected that GPT4 will also perform well.

Different types of endpoints are available. Here are a few examples for Davinci model:

1. /davinci/codex/completions: This endpoint is specifically designed for code-related completions. It utilizes the Codex model, which is trained on a large codebase and is well-suited for assisting with programming tasks.
2. /davinci/instruct/completions: This endpoint is optimized for instruction-based completions. It performs well when the prompt includes explicit instructions or specifications for the desired output.
3. /davinci/language/detection: This endpoint allows you to detect the language of a given text. It provides language identification capabilities using the Davinci model.
4. /davinci/translate: This endpoint enables translation tasks, allowing you to translate text from one language to another using the Davinci model.

We’ll always use an instruct/completions endpoint in the examples below because it is specifically designed to process direct instructions. Unlike other endpoints that are more optimized for chats, code, or translation, the instruct/completions endpoint is particularly suitable for accurately interpreting and generating responses based on explicit instructions provided to the model. This endpoint receives a prompt string with the request in natural language. Then, it responds also in a natural language string.

The model doesn’t automatically retain memory between requests, so you’ll need to include all context within the prompt for each request. For economic reasons, it is preferable to use a single request to send multiple queries, as we will see in the next sections.

Prompt Architecture

First, we need to define an architecture for our prompt.

At the beginning, we should set all pieces of text that are requests as context. Here, we will refer to them as “documents”:

Consider a document called '{DOCUMENT_1_NAME}':
"{DOCUMENT_1_TEXT}"
Consider a document called '{DOCUMENT_2_NAME}':
"{DOCUMENT_2_TEXT}"
...
Consider a document called '{DOCUMENT_N_NAME}':
"{DOCUMENT_N_TEXT}"

These documents could be, for example:

• Emails from a staff member
• Product reviews
• etc.

Next, we can instruct the model to read our JSON schema and generate responses for each section of the JSON:

Considering the above documents, generate JSON data in the following format:
{
	"question_1": "{QUESTION_1_STRING}",
	"question_2": "{QUESTION_2_STRING}",
	"question_n": "{QUESTION_N_STRING}"
}
Please ensure that the values are filled as specified.
Provide only the answer in JSON format and nothing more.

These questions should include a specific command for the AI, such as:

• Generate a short title for the document named ‘{DOCUMENT_X_NAME}’.
• Write the names of the documents, separated by commas, that contain any mention of any company name.
• Write ‘Yes’ if it makes sense for the document ‘{DOCUMENT_X_NAME}’ to have a tag called ‘{TAG_NAME}’. Write ‘No’ otherwise.

Extracting the knowledge

Since we have requested a well-known JSON format as the response, it is easy to unmarshal the content in any language. An example in Python of a class to handle this is as follows:

import json
class Questions:
    def __init__(self, question_1, question_2, question_n):
        self.question_1 = question_1
        self.question_2 = question_2
        self.question_n = question_n
def json_to_questions(json_string):
    data = json.loads(json_string)
    questions = Questions(data["question_1"], data["question_2"], data["question_n"])
    return questionsP

Conclusion

In the rapidly advancing frontier of artificial intelligence, managing AI-generated data within production systems presents both a complex challenge and an exciting opportunity. This article presents a new framework for handling AI-generated data within production systems, specifically focusing on integrating OpenAI models such as GPT-3, GPT-4, and several Davinci models.

However, it’s important to note that due to the fast-paced and uncharted nature of this field, the strategies detailed in this article may soon be outdated. To conclude, this framework should be seen as a starting point in the thrilling and ever-evolving journey of implementing AI in production systems. It’s essential to stay adaptable and prepared for a flood of future innovations.