Retrieve and Index Search Engine Data

AgentSearch Loader is a LlamaIndex reader that connects your data pipeline to hosted search APIs (including Bing and AgentSearch) and RAG-specialized language models such as SciPhi/Sensei-7B-V1. At query time, you pass a search string, choose a search provider, and specify a RAG model; the loader returns documents.

Use AgentSearch Loader when you need to augment LlamaIndex workflows with web search or specialized search datasets, especially when you want RAG-optimized LLMs to process search results. It fits scenarios where your application needs to retrieve information from external search sources.

Do not use this loader if your use case relies entirely on private, offline document collections with no need for external search APIs. Avoid it when you cannot accommodate the latency of external API calls or when your compliance requirements prohibit sending queries to third-party search services.

You provide a query string, a search provider identifier (such as bing or agent-search), and a RAG provider model name (for example, SciPhi/Sensei-7B-V1). Optionally, you can set the SCIPHI_API_KEY environment variable for authentication.

The loader returns document objects. Based on the example, documents contain a text field and a metadata dictionary.

pip install llama-index-readers-agent-search

### Optionally set the API key in the env
### import os
### os.environ["SCIPHI_API_KEY"] = "..."

from llama_index.readers.agent_search import AgentSearchReader

reader = AgentSearch()

document = reader.load_data(
    query="latest news",
)[0]
### text = "The latest news encompasses ... and its consequences [2]."
### metadata = {'related_queries': ['Details on the...', ...], 'search_results' : [...]}

AgentSearch Loader integrates with the AgentSearch dataset and hosted search APIs including Bing. It works with RAG-specialized language models such as SciPhi/Sensei-7B-V1. The loader is designed as a data source for LlamaIndex, fitting into the broader LlamaIndex ecosystem for building retrieval-augmented generation pipelines.

This loader serves developers building LlamaIndex-based applications who need to incorporate search APIs or search datasets into their RAG workflows. If you are already using LlamaIndex and need to extend your data sources beyond static documents, AgentSearch Loader provides a straightforward integration path.