Integrate OpenAI Assistants API with Python SDK

Assistants API Overview (Python SDK)

The new Assistants API is a stateful evolution of our Chat Completions API meant to simplify the creation of assistant-like experiences, and enable developer access to powerful tools like Code Interpreter and File Search.

Chat Completions API vs Assistants API

The primitives of the Chat Completions API are Messages, on which you perform a Completion with a Model (gpt-4o, gpt-4o-mini, etc). It is lightweight and powerful, but inherently stateless, which means you have to manage conversation state, tool definitions, retrieval documents, and code execution manually.

The primitives of the Assistants API are

• Assistants, which encapsulate a base model, instructions, tools, and (context) documents,
• Threads, which represent the state of a conversation, and
• Runs, which power the execution of an Assistant on a Thread, including textual responses and multi-step tool use.

We'll take a look at how these can be used to create powerful, stateful experiences.

Setup

Python SDK

Note
We've updated our Python SDK to add support for the Assistants API, so you'll need to update it to the latest version (1.59.4 at time of writing).

And make sure it's up to date by running:

Pretty Printing Helper

Complete Example with Assistants API

Assistants

The easiest way to get started with the Assistants API is through the Assistants Playground.

Let's begin by creating an assistant! We'll create a Math Tutor just like in our docs.

You can also create Assistants directly through the Assistants API, like so:

Regardless of whether you create your Assistant through the Dashboard or with the API, you'll want to keep track of the Assistant ID. This is how you'll refer to your Assistant throughout Threads and Runs.

Next, we'll create a new Thread and add a Message to it. This will hold the state of our conversation, so we don't have re-send the entire message history each time.

Threads

Create a new thread:

Then add the Message to the thread:

Note
Even though you're no longer sending the entire history each time, you will still be charged for the tokens of the entire conversation history with each Run.

Runs

Notice how the Thread we created is not associated with the Assistant we created earlier! Threads exist independently from Assistants, which may be different from what you'd expect if you've used ChatGPT (where a thread is tied to a model/GPT).

To get a completion from an Assistant for a given Thread, we must create a Run. Creating a Run will indicate to an Assistant it should look at the messages in the Thread and take action: either by adding a single response, or using tools.

Note
Runs are a key difference between the Assistants API and Chat Completions API. While in Chat Completions the model will only ever respond with a single message, in the Assistants API a Run may result in an Assistant using one or multiple tools, and potentially adding multiple messages to the Thread.

To get our Assistant to respond to the user, let's create the Run. As mentioned earlier, you must specify both the Assistant and the Thread.

Unlike creating a completion in the Chat Completions API, creating a Run is an asynchronous operation. It will return immediately with the Run's metadata, which includes a status that will initially be set to queued. The status will be updated as the Assistant performs operations (like using tools and adding messages).

To know when the Assistant has completed processing, we can poll the Run in a loop. (Support for streaming is coming soon!) While here we are only checking for a queued or in_progress status, in practice a Run may undergo a variety of status changes which you can choose to surface to the user. (These are called Steps, and will be covered later.)

Messages

Now that the Run has completed, we can list the Messages in the Thread to see what got added by the Assistant.

As you can see, Messages are ordered in reverse-chronological order - this was done so the most recent results are always on the first page (since results can be paginated). Do keep a look out for this, since this is the opposite order to messages in the Chat Completions API.

Let's ask our Assistant to explain the result a bit further!

This may feel like a lot of steps to get a response back, especially for this simple example. However, you'll soon see how we can add very powerful functionality to our Assistant without changing much code at all!

Example

Let's take a look at how we could potentially put all of this together. Below is all the code you need to use an Assistant you've created.

Since we've already created our Math Assistant, I've saved its ID in MATH_ASSISTANT_ID. I then defined two functions:

• submit_message: create a Message on a Thread, then start (and return) a new Run
• get_response: returns the list of Messages in a Thread

I've also defined a create_thread_and_run function that I can re-use (which is actually almost identical to the client.beta.threads.create_and_run compound function in our API ;) ). Finally, we can submit our mock user requests each to a new Thread.

Notice how all of these API calls are asynchronous operations; this means we actually get async behavior in our code without the use of async libraries! (e.g. asyncio)

Once all Runs are going, we can wait on each and get the responses.

Et voilà!

You may have noticed that this code is not actually specific to our math Assistant at all... this code will work for any new Assistant you create simply by changing the Assistant ID! That is the power of the Assistants API.

Tools

A key feature of the Assistants API is the ability to equip our Assistants with Tools, like Code Interpreter, File Search, and custom Functions. Let's take a look at each.

Code Interpreter

Let's equip our Math Tutor with the Code Interpreter tool, which we can do from the Dashboard...

...or the API, using the Assistant ID.

Now, let's ask the Assistant to use its new tool.

And that's it! The Assistant used Code Interpreter in the background, and gave us a final response.

For some use cases this may be enough - however, if we want more details on what precisely an Assistant is doing we can take a look at a Run's Steps.

Steps

A Run is composed of one or more Steps. Like a Run, each Step has a status that you can query. This is useful for surfacing the progress of a Step to a user (e.g. a spinner while the Assistant is writing code or performing retrieval).

Let's take a look at each Step's step_details.

We can see the step_details for two Steps:

1. tool_calls (plural, since it could be more than one in a single Step)
2. message_creation

The first Step is a tool_calls, specifically using the code_interpreter which contains:

• input, which was the Python code generated before the tool was called, and
• output, which was the result of running the Code Interpreter.

The second Step is a message_creation, which contains the message that was added to the Thread to communicate the results to the user.

File search

Another powerful tool in the Assistants API is File search. This allows the uploading of files to the Assistant to be used as a knowledge base when answering questions.

Note
There are more intricacies in File Search, like Annotations, which may be covered in another cookbook.

Functions

As a final powerful tool for your Assistant, you can specify custom Functions (much like the Function Calling in the Chat Completions API). During a Run, the Assistant can then indicate it wants to call one or more functions you specified. You are then responsible for calling the Function, and providing the output back to the Assistant.

Let's take a look at an example by defining a display_quiz() Function for our Math Tutor.

This function will take a title and an array of questions, display the quiz, and get input from the user for each:

• title
• questions
  • question_text
  • question_type: [MULTIPLE_CHOICE, FREE_RESPONSE]
  • choices: ["choice 1", "choice 2", ...]

I'll mocking out responses with get_mock_response.... This is where you'd get the user's actual input.

Here's what a sample quiz would look like:

Now, let's define the interface of this function in JSON format, so our Assistant can call it:

Once again, let's update our Assistant either through the Dashboard or the API.

Note
Pasting the function JSON into the Dashboard was a bit finicky due to indentation, etc. I just asked ChatGPT to format my function the same as one of the examples on the Dashboard :).

And now, we ask for a quiz.

Now, however, when we check the Run's status we see requires_action! Let's take a closer.

The required_action field indicates a Tool is waiting for us to run it and submit its output back to the Assistant. Specifically, the display_quiz function! Let's start by parsing the name and arguments.

Note
While in this case we know there is only one Tool call, in practice the Assistant may choose to call multiple tools.

Now let's actually call our display_quiz function with the arguments provided by the Assistant:

Great! (Remember these responses are the one's we mocked earlier. In reality, we'd be getting input from the back from this function call.)

Now that we have our responses, let's submit them back to the Assistant. We'll need the tool_call ID, found in the tool_call we parsed out earlier. We'll also need to encode our listof responses into a str.

We can now wait for the Run to complete once again, and check our Thread!

Woohoo 🎉

Conclusion

We covered a lot of ground in this notebook, give yourself a high-five! Hopefully you should now have a strong foundation to build powerful, stateful experiences with tools like Code Interpreter, Retrieval, and Functions!

There's a few sections we didn't cover for the sake of brevity, so here's a few resources to explore further:

• Annotations: parsing file citations
• Files: Thread scoped vs Assistant scoped
• Parallel Function Calls: calling multiple tools in a single Step
• Multi-Assistant Thread Runs: single Thread with Messages from multiple Assistants
• Streaming: coming soon!

Now go off and build something amazing!