OpenAI Assistants API v2: What’s New and Improved?

Artificial intelligence is rapidly transforming the way we interact with technology. AI-powered assistants are at the forefront of this revolution, and with the latest update to OpenAI’s Assistants API v2, these virtual helpers are getting even smarter and more versatile.

Artificial intelligence is rapidly transforming the way we interact with technology. OpenAI has updated their Assistants API and with came a bunch of changes. Here are a few of them:

1. Improved Retrieval Tool: File_Search

One of the most significant advancements in the OpenAI Assistants v2 update is the introduction of the file_search tool. This feature improves on the previous retrieval tool and revolutionizes how assistants interact with external knowledge sources. Here's why it's a big deal:

• Massive Capacity Upgrade: Assistants can now draw information from up to 10,000 files via the new vector store, 500x more than before! This empowers businesses to equip their assistants with vast libraries of product information, internal documentation, customer records, and more.
• Efficiency Boost: File_search optimizes retrieval with faster processing and multi-threaded searches. Imagine your assistant simultaneously combing through numerous documents to pinpoint the perfect answer – that's the power of parallel processing.
• Smarter Results: Advanced reranking and query rewriting techniques prioritize the most relevant content. File_search doesn't just find information, it understands the nuances of user inquiries for exceptional accuracy.
• Seamless Integration: File_search works in tandem with the new vector store concept. Upon uploading files, OpenAI automatically parses, chunks, and creates embeddings, making them immediately searchable. This eliminates manual data preparation, saving development time and effort.

How to use the new file_search tool:
To use the file_search tool, simply create an assistant with the file_search enabled in the tool parameter of the Assistant. The script should look like this.

from openai import OpenAI
 
client = OpenAI()
 
assistant = client.beta.assistants.create(
  name="Financial Analyst Assistant",
  instructions="You are an expert financial analyst. Use you knowledge base to answer questions about audited financial statements.",
  model="gpt-4-turbo",
  tools=[{"type": "file_search"}],
)

2. New Vector Store

Before the OpenAI Assistants v2 update, retrievals for assistants were handled less efficiently and with some limitations. Assistants v2 introduces vector stores to simplify data management and streamline data between each assistants. Here’s a breakdown of the differences:

• New Vector Database: Before the update users had to use a third party database to store their files since Assistants can only use up to 20, the new update increased that up to 10,000 files by adding a new vector storage feature.
• Automatic Processing: Files are parsed, chunked, and embedded (turned into numerical representations) automatically upon upload.
• Shared Across Assistants: Vector stores can be used by multiple assistants, reducing redundancy.

How Things Worked with Assistants v1:

• More Limited Retrieval: Assistants could access information, but the process was less streamlined and scalable.
• Lack of Dedicated Tools: There was no equivalent to file_search, meaning retrieval techniques were less sophisticated.
• No Vector Store Concept: Data had to be embedded and managed more manually, which was often a complex overhead.

How to use the new vector_store :
You can create a vector store and add files to it in a single API call, it should look like this.

vector_store = client.beta.vector_stores.create(
  name="Product Documentation",
  file_ids=['file_1', 'file_2', 'file_3', 'file_4', 'file_5']
)

Files can also be added to a vector store after its created by following the script below.

vector_store_file = client.beta.vector_stores.files.create(
  vector_store_id="vs_abc123",
  file_id="file-abc123"
)
print(vector_store_file)

If you would like further assistance with the new Assistant V2 Vector Stores or need help integrating this into your business, feel free to schedule a consultation with us at www.woyera.com.

3. Enhanced User Control

The OpenAI Assistants v2 update puts greater control into the hands of developers and users with the addition of the new tool choice parameter, token control, and the popular model configuration parameters.

Tool Choice: The tool_choice parameter has taken the place of the previously used function_call. Tool choice controls which tools such as file_search, code_interpreter, or a function in a specific run.

• If you want the model to pick between generating a message or calling on the tools available, set to "tool_choice": "auto" and if you don’t want the model to call any tools set it to "tool_choice": "none" .
• If you wanted the run to use a specified tool, it will look like the following:{"type": "file_search"} or {"type": "function", "function": {"name": "my_function"}} .

Control Max Tokens: OpenAI assistants now offer more refined token controls. ‘Control max tokens’ lets you set limits on the number of tokens an assistant can use in a response. This allows for cost optimization since OpenAI pricing is partly based on token usage, these controls help manage costs and reduce unnecessary spending.

Model Configuration Parameters: The new v2 update has also added the popular parameter configurations used in LLM’s such as Temperature, Top_p (nucleus sampling) and response_format.

4. Fine-Tuned Models and Streaming Response

The update has also made life easier for both developers and users, the developers are now able to use fine-tuned models for the Assistants API, while the users are now able to see a streamed response from the Assistants.

Streaming Responses: Streaming in Assistants v2 allows Assistants to start sending text fragments as soon as they are generated. Users perceive responses as being delivered faster, making the interaction feel more responsive and have a more natural feel.

Fine-Tuned Models: You can now use fine-tuned models for the Assistants API, although only fine-tuned versions of gpt-3.5-turbo-0125 are supported. Using a fine tuned model for your assistant is simple, fill the “model” line to the model fine-tuned model you want to use following the example below.

from openai import OpenAI
client = OpenAI()

completion = client.chat.completions.create(
  model="ft:gpt-3.5-turbo:my-org:custom_suffix:id",
  messages=[
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Hello!"}
  ]
)
print(completion.choices[0].message)

These enhancements in the Assistants API v2 showcase OpenAI’s continuous efforts to empower developers with sophisticated, yet accessible AI tools. Be aware that this update is still in Beta so expect a few bugs to occur.

If you need advice on how to set up OpenAI Assistants v2 or to see if its right for you please schedule a FREE 30-minute call at www.woyera.com.