[Image by DALL-E and Author]

Getting started with OpenAI’s Chat Completions API in 2024

A Guide to Creating Your Own Simple Chatbot

The different APIs that OpenAI offers each have their own role in today’s AI ecosystem. The Chat Completion API is one of these tools. In the simplest terms, this API connects users to the most basic and fundamental features of ChatGPT. It acts as the bridge that connects applications to people, enabling the generation of human-like text responses based on user input. This API is the core building block of potential AI products from customer service to sales all of which require fast and accurate response in a natural way.

In this post, I will cover:

• Getting Started with Chat Completions
• Features of the API
• Templates for beginners
• Use cases

Getting Started with Chat Completions

When using the Chat Completions API you’ll start with code that looks like what is shown below. Let’s break down what’s happening.

from openai import OpenAI
client = OpenAI()

response = client.chat.completions.create(
  model="gpt-3.5-turbo",
  messages=[
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Who won the world series in 2020?"},
    {"role": "assistant", "content": "The Los Angeles Dodgers won the World Series in 2020."},
    {"role": "user", "content": "Where was it played?"}
  ]
)

The “messages” parameter is where the model can be customized. This parameter consists of two pieces of information, the “role” and the “content”. You can see that three unique roles are present in this example.

• “role”: “system”: The “system” role establishes the behavior of the chatbot. You can change the actions and personality by modifying this code. Interestingly, this is an optional feature that will default to “You are a helpful assistant” unless specified differently. There will only ever be one system message.
• “role”: “user” : The “user” role is assigned to the messages that the user(s) gives the chatbot to respond to. Please note that the example above has two unique user inputs, but yours could have none. Alternatively, you could include an example user prompt for the system to look at and use for future reference.
• “role”: “assistant”: The “assistant” role will be populated with the responses that your chatbot (the API) provides to your user. Similar to the “user” messages, you can have an example message that will be used to help the API respond appropriately. All previous “assistant” messages can be manually stored and referred to for future responses if specified.

Now that we have an understanding of the roles, let’s look at the content.

• “content”: As the name suggests, the content is the query / response that is associated with each role. The API and model will interact with the content of each message.

Now lets see what a response from the API looks like.

{
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "message": {
        "content": "The 2020 World Series was played in Texas at Globe Life Field in Arlington.",
        "role": "assistant"
      },
      "logprobs": null
    }
  ],
  "created": 1677664795,
  "id": "chatcmpl-7QyqpwdfhqwajicIEznoc6Q47XAyW",
  "model": "gpt-3.5-turbo-0613",
  "object": "chat.completion",
  "usage": {
    "completion_tokens": 17,
    "prompt_tokens": 57,
    "total_tokens": 74
  }
}

Here you can find a variety of crucial information. Let’s explore what it means.

• “usage”: Tokens are an essential part of all LLMs (Large Language Models). The Chat Completion API is no different. Not only are you charged based on the number of tokens used in BOTH the prompt and response, but all the fields they include (role, content, ect). Whether this is your first time hearing the term or you’ve worked with them before, I highly recommend that you check out the OpenAI documentation about tokens. Tokens could be an article of their own so I won’t go too deep into the topic.
• “finish_reason”: Every API call will have a finish reason. These can include; “stop”, “length”, “function_call”, “content_filter”, and “null”. You can see the OpenAI docs for the full details, but you’ll know that no errors occurred if you have “stop” or “funciton_call” listed as your finish reason.
• “message”: This message includes both the content (the response) and the role (who is giving the response). We’ll see below how you can extract this data for use in a chatbot.

The last important element to understand is the format of the response. In order to return just the assistant’s response to the user, you should use a variation of the response shown below.

response['choices'][0]['message']['content]

Features of the API

Now that we have a basic starting point for working with the API, let’s dive into two newer features that the Chat Completion API offers.

• JSON mode: Because of the wide range of use cases for this API, OpenAI has created the option to specify the format of each response. You can tell the API to always return a JSON object by editing the system message and by specifying the response_format type. Please note that it’s possible for invalid JSON objects to be created as a response, even if you take the steps mentioned above.
• Reproducible Responses: The Chat Completion API is non-deterministic which means that each response to the same identical question could and probably will be different. However, OpenAI has created two tools to help make responses more deterministic if desired. The simplest way is to set the seed parameter to a specific value and then apply that seed value to all future API calls. Like all software, the API is constantly being improved upon and updated. These updates may impact how deterministic your responses are, so in order to help keep track of the responses your application is creating, use the “system_fingerprint” parameter which will tell you when the underlying system might have changed which can cause results to not be reproducible anymore.

Templates for beginners

Luckily, people much smarter than myself have created a template and tutorial that can be used to get your first API powered applications up and running. I’ve found two that work particularly well, the official OpenAI quickstart guide that is available on GitHub and an additional GitHub repo that wasn’t created by OpenAI. Both of these will allow you to build a basic chat bot. Here’s what the simple (non OpenAI) example looks like when it’s running.

I was able to get this example up and running in less than an hour!

These examples can easily be built upon to create real world products. One potential avenue for development is connecting to other APIs. By incorporating the Keymate.AI API for example, you can access Keymate’s long term memory databases or use their Google browsing endpoints for real time information retrieval. Now you’ve taken a quick and easy to create sample and turned it into a marketable and useful product.

Use Cases

Now that we’ve covered what the Chat Completions API is and how to use it, the final step is to understand what can be achieved with it. Like most general AI chatbots, there are many potential use cases. One of the most useful and widely available applications is customer service chatbots. By setting the right system messages and information, a simple chatbot can help answer customer questions quickly and correctly. We are also seeing this technology in product recommendations. By explaining through conversation what you’re looking for and your budget, an application can give suggestions. While a bit more complex, an interesting use case would be connecting your chat completions app to another database and API to create a research assistant. Whether searching for academic articles or just browsing the web, research through conversation can be a powerful tool. This could be achieved by connecting to services like Keymate.AI.