Orchestrate Multi-Tool Responses with RAG
An OpenAI cookbook routing queries between web search and a Pinecone vector database via the Responses API's multi-tool orchestration.
Why it matters
Dynamically route user queries to the most appropriate tools, including web search and vector databases, to generate context-aware responses.
Outcomes
What it gets done
Implement RAG for intelligent tool selection.
Integrate with external vector databases like Pinecone.
Orchestrate sequential tool calls for complex queries.
Leverage OpenAI's Responses API for dynamic response generation.
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/oai-responsesapitoolorchestration | bash Steps
Steps in the chain
Overview
Responses Api Tool Orchestration
An OpenAI cookbook demonstrating RAG-based multi-tool orchestration via the Responses API, routing queries between a built-in web search tool and a Pinecone vector-database lookup. Use it as a reference for building an agent that dynamically chooses between web search and an external vector database based on the query.
What it does
An OpenAI cookbook demonstrating multi-tool orchestration with a Retrieval-Augmented Generation (RAG) approach via the Responses API: it routes each user query to the right tool - a built-in web search for general or explicitly internet-flagged questions, or a Pinecone vector-database lookup for domain-specific internal context - rather than using one fixed tool for everything. The example uses a medical reasoning dataset from Hugging Face, merging its "Question" and "Response" columns into embedded text stored as Pinecone metadata, to demonstrate retrieving medical-literature-style context.
When to use - and when NOT to
Use it as a reference when building an agent that needs to dynamically choose between multiple retrieval sources - the Responses API's built-in web search versus an external vector database like Pinecone - based on what a given query actually needs. For RAG specifically over PDFs via the Responses API's built-in file_search tool, the cookbook points to a companion notebook instead of duplicating that flow here.
Inputs and outputs
The pipeline: convert a dataset (here, Hugging Face's medical reasoning set) into a Pandas DataFrame, merge the relevant text columns, and use one sample embedding to size a Pinecone index; upsert the full dataset in batches with Question/Answer metadata attached to each vector, with the option to update metadata on specific entries afterward; at query time, embed the incoming natural-language question and run a similarity search against the index, returning metadata for context. Two tools are defined for the model to call: a Web Search Preview Tool for live, real-time internet lookups, and a Pinecone Search Tool (PineconeSearchDocuments, backed by query_pinecone_index) for semantic search over the stored medical content. Depending on the query, the model calls web_search_call for general/non-health or explicit internet-search requests, PineconeSearchDocuments for domain-specific questions, or answers directly with no tool call; the tool's output is appended to the conversation before the Responses API generates the final answer.
The cookbook then walks through several example queries to show each routing branch firing correctly, and demonstrates modifying the input query and the system instructions passed to the Responses API to steer a specific tool-calling sequence.
Integrations
Built on OpenAI's Responses API, with Pinecone as the external vector database and Hugging Face as the dataset source; also demonstrates the Responses API's built-in web-search tool alongside the custom Pinecone function tool in the same orchestration flow.
Who it's for
Developers building RAG or agentic applications who need a query router that picks between general web search and a domain-specific vector database automatically, rather than hardcoding one retrieval path for every query.
Source README
Multi-Tool Orchestration with RAG approach using OpenAI's Responses API
This cookbook guides you through building dynamic, multi-tool workflows using OpenAI's Responses API. It demonstrates how to implement a Retrieval-Augmented Generation (RAG) approach that intelligently routes user queries to the appropriate in-built or external tools. Whether your query calls for general knowledge or requires accessing specific internal context from a vector database (like Pinecone), this guide shows you how to integrate function calls, web searches in-built tool, and leverage document retrieval to generate accurate, context-aware responses.
For a practical example of performing RAG on PDFs using the Responses API's file search feature, refer to this notebook.
This example showcases the flexibility of the Responses API, illustrating that beyond the internal file_search tool-which connects to an internal vector store-there is also the capability to easily connect to external vector databases. This allows for the implementation of a RAG approach in conjunction with hosted tooling, providing a versatile solution for various retrieval and generation tasks.
In this example we use a sample medical reasoning dataset from Hugging Face. We convert the dataset into a Pandas DataFrame and merge the “Question” and “Response” columns into a single string. This merged text is used for embedding and later stored as metadata.
Create a Pinecone Index Based on the Dataset
Use the dataset itself to determine the embedding dimensionality. For example, compute one embedding from the merged column and then create the index accordingly.
Upsert the Dataset into Pinecone index
Process the dataset in batches, generate embeddings for each merged text, prepare metadata (including separate Question and Answer fields), and upsert each batch into the index. You may also update metadata for specific entries if needed.
Query the Pinecone Index
Create a natural language query, compute its embedding, and perform a similarity search on the Pinecone index. The returned results include metadata that provides context for generating answers.
Generate a Response Using the Retrieved Context
Select the best matching result from your query results and use the OpenAI Responses API to generate a final answer by combining the retrieved context with the original question.
Orchestrate Multi-Tool Calls
Now, we'll define the built-in function available through the Responses API, including the ability to invoke the external Vector Store - Pinecone as an example.
Web Search Preview Tool: Enables the model to perform live web searches and preview the results. This is ideal for retrieving real-time or up-to-date information from the internet.
Pinecone Search Tool: Allows the model to query a vector database using semantic search. This is especially useful for retrieving relevant documents-such as medical literature or other domain-specific content-that have been stored in a vectorized format.
As shown above, depending on the query, appropriate tool is invoked in order to determine the optimal response.
For instance, in the third example, when the model calls PineconeSearchDocuments, the code passes the current query to query_pinecone_index and returns the best match as context. For general or non-health questions, or when the user explicitly asks for an internet search, the code calls web_search_call. For other questions, the model may answer without a tool.
Finally, the tool call and its output are appended to the conversation, and the final answer is generated by the Responses API.
Multi-tool orchestration flow
Now let us try to modify the input query and the system instructions to the responses API in order to follow a tool calling sequence and generate the output.
Here, we used OpenAI's Responses API to implement Retrieval-Augmented Generation (RAG) with multiple tools. The model selects a tool based on the query: built-in web search can handle general questions, while function calls can retrieve internal medical context from a vector database such as Pinecone. We also combined multiple tool calls in sequence to generate a final response from the instructions provided to the Responses API.
As you continue to experiment and build upon these concepts, consider exploring additional resources and examples to further enhance your understanding and applications
Happy coding!
FAQ
Common questions
Discussion
Questions & comments · 0
Sign In Sign in to leave a comment.