ChatGPT App Store for AI Plugins

Access ChatGPT Plugins TODAY with LangChain

With the introduction of support for plugins in ChatGPT, OpenAI is looking to create its iPhone App Store moment.

Here’s what we’ll cover in this article:

• What is a ChatGPT Plugin?
• App Store (for ChatGPT Plugins)
• ChatGPT Plugin Example: Klarna Shopping
• The OpenAPI Specification
• ChatGPT Plugins with LangChain
• The Future for ChatGPT Plugins

One of the primary challenges faced by ChatGPT is that its training data remains current only up to September 2021; however, the advent of ChatGPT plugins, such as Browsing, has opened up a new realm of possibilities. These plugins enable users to browse the internet and gather real-time information pertinent to their conversation prompts, effectively overcoming this limitation and enhancing the overall ChatGPT experience.

What is a ChatGPT Plugin?

Currently only available in a limited alpha release. Join the waitlist.

Plugins are tools that help improve ChatGPT’s abilities. They gather up-to-date information and link with outside services, letting users interact with different platforms like online shops, calculators, blogs, and web apps using ChatGPT.

For developers, plugins are OpenAPI specs that connect to APIs.

You can also think of plugins as you do apps on your smartphone. The apps extend the capabilities of your phone and can be installed via an App Store.

App Store (for ChatGPT Plugins)

Here’s a sampling of the initial ChatGPT plugins available for install today.

The Plugin Store

Creating an app store offers several primary benefits for developers, users, and platform owners:

1. Centralized marketplace: An app store provides a centralized platform for developers to distribute their applications and for users to discover and download them.
2. Monetization opportunities: App stores enable developers to monetize their applications through various business models, including paid plugins, in-app purchases, subscriptions, and advertising.
3. Analytics and insights: App stores provide developers with valuable analytics and insights on app performance, user engagement, and revenue. This data helps developers understand how their plugins are performing, identify areas for improvement, and make data-driven decisions to optimize their plugins’ success.

By creating an app store for plugins, developers, users, and platform owners can benefit from a more streamlined, secure, and potentially lucrative ecosystem that fosters innovation and growth.

To begin developing your own ChatGPT plugins, let’s take a look at one of the initial plugins, Klarna Shopping.

ChatGPT Plugin Example: Klarna Shopping

https://www.klarna.com/.well-known/ai-plugin.json

{
  "schema_version": "v1",
  "name_for_model": "KlarnaProducts",
  "name_for_human": "Klarna Shopping",
  "description_for_human": "Search and compare prices from thousands of online shops",
  "description_for_model": "Use the Klarna plugin to get relevant product suggestions for any shopping or researching purpose. The query to be sent should not include stopwords like articles, prepositions and determinants. The api works best when searching for words that are related to products, like their name, brand, model or category. Links will always be returned and should be shown to the user.",
  "api": {
    "type": "openapi",
    "url": "https://www.klarna.com/us/shopping/public/openai/v0/api-docs/",
    "has_user_authentication": false
  },
  "auth": {
    "type": "none"
  },
  "logo_url": "https://www.klarna.com/assets/sites/5/2020/04/27143923/klarna-K-150x150.jpg",
  "contact_email": "openai-products@klarna.com",
  "legal_info_url": "https://www.klarna.com/us/legal/"
}

ChatGPT plugin files contain metadata and configuration information for an AI plugin. Here’s a breakdown of each field:

1. "schema_version": "v1": Specifies the version of the plugin schema used in this JSON file, which is version 1.
2. "name_for_model": "KlarnaProducts": This is the name used by the AI model to identify and access the plugin.
3. "name_for_human": "Klarna Shopping": This is the user-friendly name of the plugin, intended for human users.
4. "description_for_human": "Search and compare prices from thousands of online shops": This is a short description of the plugin's purpose, intended for human users.
5. "description_for_model": "Use the Klarna plugin...": This is a description of the plugin's functionality and instructions for the AI model on how to use it.
6. "api": {...}: This object contains information about the API associated with the plugin.
7. "type": "openapi": Specifies the API type, which is OpenAPI in this case. We will discuss the OpenAPI Specification shortly.
8. "url": "https://www.klarna.com/us/shopping/public/openai/v0/api-docs/": The URL where the API documentation (which adheres to the OpenAPI Spec) can be found.
9. "has_user_authentication": false: Indicates whether the API requires user authentication. In this case, it does not.
10. "auth": {...}: Contains information about the plugin's authentication method.
11. "type": "none": Specifies that no authentication is required for this plugin.
12. "logo_url": "https://www.klarna.com/assets/sites/5/2020/04/27143923/klarna-K-150x150.jpg": The URL where the plugin's logo can be found.
13. "contact_email": "openai-products@klarna.com": The contact email address for inquiries related to the plugin.
14. "legal_info_url": "https://www.klarna.com/us/legal/": The URL where legal information related to the plugin can be found.

This JSON file serves as a configuration and metadata file for the Klarna Shopping ChatGPT plugin, providing both the AI model and human users with necessary information about the plugin’s purpose, usage, contract details, and OpenAPI Specification.

The OpenAPI Specification

The OpenAPI Specification (OAS), formerly known as Swagger, is a widely adopted, standardized API description format for RESTful APIs. It provides a human-readable and machine-readable way to describe, document, and share the details of an API’s endpoints, request and response formats, authentication, and other key information. By using the OpenAPI Specification, developers can create and consume APIs more efficiently, while tools can generate code, documentation, and tests automatically.

The OpenAPI Specification consists of several core components, which are described in detail below:

openapi: 3.0.1
info:
  title: TODO Plugin
  description: A plugin that allows the user to create and manage a TODO list using ChatGPT.
  version: 'v1'
servers:
  - url: http://localhost:3000
paths:
  /todos:
    get:
      operationId: getTodos
      summary: Get the list of todos
      responses:
        "200":
          description: OK
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/getTodosResponse'
components:
  schemas:
    getTodosResponse:
      type: object
      properties:
        todos:
          type: array
          items:
            type: string
          description: The list of todos.

OpenAPI Object: The root document object of the API specification. It contains general information about the API, like its version, title, description, and other metadata. The OpenAPI object includes the following main properties:

• openapi: Specifies the version of the OAS being used (i.e. “3.0.1”).
• info: Contains metadata about the API, such as its title, description, version, terms of service, contact information, and license.
• servers: An array of Server Objects that define the base URL and other details of the API’s servers.
• paths: Describes the available paths and operations (e.g. GET, PUT, POST, DELETE) for the API.
• components: A container for reusable components like schemas, responses, parameters, request bodies, and security schemes.
• security: Specifies the security requirements for the API.
• tags: An array of Tag Objects that can be used for organizing and documenting the API’s operations.

How can we connect to ChatGPT plugins TODAY — even if we are NOT one of the limited alpha release testers?

Enter LangChain — a framework for developing applications powered by language models.

With LangChain, we will connect to ChatGPT plugins and allow our Large Language Model (LLM) to interpret the plugin’s OpenAPI Spec and communicate with a third-party service (i.e. Klarna Shopping).

ChatGPT Plugins with LangChain

The following code demonstrates how to create a custom ChatOpenAI instance, load AI plugins, and execute a specific agent chain to process an input question using the OpenAI API and ChatGPT plugins.

Currently, LangChain is only able to connect to plugins with no authentication.

1. llm = ChatOpenAI(temperature=0, model_name="gpt-3.5-turbo"): This line initializes a ChatOpenAI instance with the specified model name (gpt-3.5-turbo) and temperature (0). The temperature controls the randomness of the AI's output; a lower value like 0 makes the AI more deterministic and focused.
2. tool = AIPluginTool.from_plugin_url("https://www.klarna.com/.well-known/ai-plugin.json"): This line creates an AIPluginTool object by loading the AI plugin JSON file from Klarna's website.
3. tools = load_tools(["requests"]): This line loads a list of tools, in this case, only the "requests" tool. This is a built-in tool that allows the AI to make HTTP requests.
4. tools += [tool]: This line adds the Klarna plugin (tool) to the list of available tools.
5. The PREFIX and SUFFIX variables define the prompt format for the AI. The PREFIX variable informs the AI that it has access to the specified tools and should answer the questions as best as it can. The SUFFIX variable structures the input question and response using the agent's scratchpad for thoughts.
6. agent_arguments_prefix_suffix = {"prefix": PREFIX, "suffix": SUFFIX}: This line creates a dictionary containing the prefix and suffix arguments to be passed to the agent during initialization.
7. agent_chain = initialize_agent(tools, llm, agent="zero-shot-react-description", verbose=True, agent_kwargs=agent_arguments_prefix_suffix): This line initializes an agent chain with the specified tools, the ChatOpenAI instance, and the agent type (zero-shot-react-description). The verbose=True argument enables detailed output during execution, while agent_kwargs passes the prefix and suffix arguments to the agent.
8. agent_chain.run("what panini cards are available on klarna?"): This line runs the agent chain with the input question, "what panini cards are available on klarna?" The agent chain processes the question, utilizing the available tools (including the Klarna plugin) and the specified AI model to generate a response.

In summary, this code sets up a custom ChatOpenAI instance, loads a Klarna ChatGPT plugin, and initializes an agent chain to process a user’s input question using the specified LLM, tools, and prompt format.

The Future for ChatGPT Plugins

Just as the first generation iPhone App Store was limited, so is the first batch of ChatGPT plugins. However, if we look around the corner, here are some plugins that I expect to become available to ChatGPT users and OpenAI API developers:

1. A plugin that will allow ChatGPT to generate images from text. Midjourney being accessible through ChatGPT would be amazing.
2. If we are talking generative art, why not a plugin to generate videos from text. Runwayml would be a prime candidate.
3. Data visualizations are ripe for a plugin. While Wolfram Alpha is part of the initial batch of plugins, we can expect more sophisticated 3D visualizations, like Apache ECharts, to eventually become integrated.
4. With Wolfram Alpha available, Wikipedia must now be anticipated.
5. A Python code interpreter, alongside browsing, is a plugin made available by OpenAI on day one; however, a code interpreter across all programming languages (e.g. JavaScript, Java, C++, etc…) and scripting for cloud platforms (i.e. AWS, GCP, and Azure) feels like an inevitability.

Looking further ahead, the ChatGPT plugin ecosystem will grow and develop quickly, offering new opportunities and creativity.

Developers will make more varied plugins for better AI interactions with different services and platforms.

Users can expect more customization, enhanced user experiences, and greater utility as AI-powered communication becomes an integral part of our daily lives.

The growing adoption of ChatGPT plugins will pave the way for the next generation of AI-driven solutions, transforming the way we work, learn, and engage with technology.

Resources

GPT-4 API Reference Guide

GPT-4 API Pricing Analysis

Create a ChatGPT Discord Bot

ChatGPT plugins

ChatGPT Plugins

OpenAPI Specification