Orchestrate Reasoning Models with Function Calls

Managing Function Calls With Reasoning Models

OpenAI now offers function calling using reasoning models. Reasoning models are trained to follow logical chains of thought, making them better suited for complex or multi-step tasks.

Reasoning models like o3 and o4-mini are LLMs trained with reinforcement learning to perform reasoning. Reasoning models think before they answer, producing a long internal chain of thought before responding to the user. Reasoning models excel in complex problem solving, coding, scientific reasoning, and multi-step planning for agentic workflows. They're also the best models for Codex CLI, our lightweight coding agent.

For the most part, using these models via the API is very simple and comparable to using familiar 'chat' models.

However, there are some nuances to bear in mind, particularly when it comes to using features such as function calling.

All examples in this notebook use the newer Responses API which provides convenient abstractions for managing conversation state. However the principles here are relevant when using the older chat completions API.

Making API calls to reasoning models

Let's make a simple call to a reasoning model using the Responses API.
We specify a low reasoning effort and retrieve the response with the helpful output_text attribute.
We can ask follow up questions and use the previous_response_id to let OpenAI manage the conversation history automatically

Nice and easy!

We're asking relatively complex questions that may require the model to reason out a plan and proceed through it in steps, but this reasoning is hidden from us - we simply wait a little longer before being shown the response.

However, if we inspect the output we can see that the model has made use of a hidden set of 'reasoning' tokens that were included in the model context window, but not exposed to us as end users.
We can see these tokens and a summary of the reasoning (but not the literal tokens used) in the response.

It is important to know about these reasoning tokens, because it means we will consume our available context window more quickly than with traditional chat models.

Calling custom functions

What happens if we ask the model a complex request that also requires the use of custom tools?

• Let's imagine we have more questions about Olympic Cities, but we also have an internal database that contains IDs for each city.
• It's possible that the model will need to invoke our tool partway through its reasoning process before returning a result.
• Let's make a function that produces a random UUID and ask the model to reason about these UUIDs.

We didn't get an output_text this time. Let's look at the response output

Along with the reasoning step, the model has successfully identified the need for a tool call and passed back instructions to send to our function call.

Let's invoke the function and send the results to the model so it can continue reasoning.
Function responses are a special kind of message, so we need to structure our next message as a special kind of input:

{
    "type": "function_call_output",
    "call_id": function_call.call_id,
    "output": tool_output
}

This works great here - as we know that a single function call is all that is required for the model to respond - but we also need to account for situations where multiple tool calls might need to be executed for the reasoning to complete.

Let's add a second call to run a web search.

OpenAI's web search tool is not available out of the box with reasoning models (as of May 2025 - this may soon change) but it's not too hard to create a custom web search function using 4o mini or another web search enabled model.

Executing multiple functions in series

Some OpenAI models support the parameter parallel_tool_calls which allows the model to return an array of functions which we can then execute in parallel. However, reasoning models may produce a sequence of function calls that must be made in series, particularly as some steps may depend on the results of previous ones.
As such, we ought to define a general pattern which we can use to handle arbitrarily complex reasoning workflows:

• At each step in the conversation, initialise a loop
• If the response contains function calls, we must assume the reasoning is ongoing and we should feed the function results (and any intermediate reasoning) back into the model for further inference
• If there are no function calls and we instead receive a Reponse.output with a type of 'message', we can safely assume the agent has finished reasoning and we can break out of the loop

Now let's demonstrate the loop concept we discussed before.

Manual conversation orchestration

So far so good! It's really cool to watch the model pause execution to run a function before continuing.
In practice the example above is quite trivial, and production use cases may be much more complex:

• Our context window may grow too large and we may wish to prune older and less relevant messages, or summarize the conversation so far
• We may wish to allow users to navigate back and forth through the conversation and re-generate answers
• We may wish to store messages in our own database for audit purposes rather than relying on OpenAI's storage and orchestration
• etc.

In these situations we may wish to take full control of the conversation. Rather than using previous_message_id we can instead treat the API as 'stateless' and make and maintain an array of conversation items that we send to the model as input each time.

This poses some Reasoning model specific nuances to consider.

• In particular, it is essential that we preserve any reasoning and function call responses in our conversation history.
• This is how the model keeps track of what chain-of-thought steps it has run through. The API will error if these are not included.

Let's run through the example above again, orchestrating the messages ourselves and tracking token usage.

Note that the code below is structured for readibility - in practice you may wish to consider a more sophisticated workflow to handle edge cases

Summary

In this cookbook, we identified how to combine function calling with OpenAI's reasoning models to demonstrate multi-step tasks that are dependent on external data sources., including searching the web.

Importantly, we covered reasoning-model specific nuances in the function calling process, specifically that:

• The model may choose to make multiple function calls or reasoning steps in series, and some steps may depend on the results of previous ones
• We cannot know how many of these steps there will be, so we must process responses with a loop
• The responses API makes orchestration easy using the previous_response_id parameter, but where manual control is needed, it's important to maintain the correct order of conversation item to preserve the 'chain-of-thought'

The examples used here are rather simple, but you can imagine how this technique could be extended to more real-world use cases, such as:

• Looking up a customer's transaction history and recent correspondence to determine if they are eligible for a promotional offer
• Calling recent transaction logs, geolocation data, and device metadata to assess the likelihood of a transaction being fraudulent
• Reviewing internal HR databases to fetch an employee’s benefits usage, tenure, and recent policy changes to answer personalized HR questions
• Reading internal dashboards, competitor news feeds, and market analyses to compile a daily executive briefing tailored to their focus areas