Spark
Sign In
Tool

Augment LLM Agents with Web Search and Extraction

Parallel AI Tool integrates LlamaIndex agents with Parallel AI's Search and Extract APIs to perform web research and extract clean, LLM-optimized content

Get this
Works with llama indexparallel ai
LlamaIndex
Own this? Claim it
79
Spark score
out of 100
Updated 2 days ago
Version 0.14.23
Models
gpt 4ollama 3
Add to Favorites

Why it matters

Empower your LLM agents with advanced web research capabilities. This tool integrates with Parallel AI's Search and Extract APIs to efficiently gather and process information from the web, optimizing it for LLM consumption.

Outcomes

What it gets done

01

Perform targeted web searches using natural language objectives or keywords.

02

Extract clean, LLM-optimized content from web pages, including dynamic sites and PDFs.

03

Structure search results and extracted content for seamless integration into LLM workflows.

04

Automate web research tasks for enhanced agent intelligence.

Install

Add it to your toolbox

Run in your project directory:

curl -fsSL https://spark.entire.vc/get/li-tool-tools-parallel-web-systems | bash

Overview

Parallel AI Tool

Parallel AI Tool integrates LlamaIndex with Parallel AI's Search and Extract APIs. The Search API returns structured, compressed excerpts from web search results optimized for LLM consumption. The Extract API converts public URLs into clean, LLM-optimized content, handling JavaScript-heavy pages and PDFs. Use this tool when building LLM agents that need to perform web research, answer questions requiring current web data, or extract focused content from multiple URLs. It's ideal for creating agents that can search with natural-language objectives and retrieve structured results optimized for LLM processing.

What it does

Parallel AI Tool provides integration between LlamaIndex and Parallel AI's Search and Extract APIs, enabling LLM agents to perform web research and content extraction. The Search API returns structured, compressed excerpts from web search results optimized for LLM consumption, while the Extract API converts public URLs into clean, LLM-optimized content including JavaScript-heavy pages and PDFs.

When to use - and when NOT to

Use this tool when you need LLM agents to perform web research with structured results, extract content from JavaScript-heavy pages or PDFs, or retrieve focused excerpts based on natural-language objectives. It's ideal for building agents that answer questions requiring current web data, like "What was the GDP of France in 2023?" or extracting specific information from multiple web sources.

Do not use this tool if you only need basic web scraping without LLM optimization, or if you're working with private/authenticated content that isn't publicly accessible via URLs.

Inputs and outputs

For the search function, you provide either a natural-language objective ("What are the latest developments in renewable energy?") or traditional keyword search queries (max 5), along with optional parameters like max_results (1-40), mode (one-shot or agentic), and excerpt settings. The function returns structured documents with metadata including title, URL, and LLM-optimized excerpts.

For the extract function, you provide a list of URLs and optionally a natural-language objective or search queries to focus extraction. You can configure whether to include excerpts, full content, and character limits. The function returns documents with clean content and metadata.

Integrations

This tool integrates with LlamaIndex's FunctionAgent and agent workflow system, allowing you to combine it with any LlamaIndex-compatible LLM like OpenAI's GPT-4. It connects to Parallel AI's platform APIs for search and extraction capabilities.

Who it's for

This tool is for developers building LLM agents that need web research capabilities, data engineers extracting structured content from multiple web sources, and AI application builders who want to give their agents access to current web information. It's particularly useful for teams already using LlamaIndex who want to add robust web search and extraction without building custom scrapers.

Installation and usage

pip install llama-index-tools-parallel-web-systems

from llama_index.tools.parallel_web_systems import ParallelWebSystemsToolSpec
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.llms.openai import OpenAI

# Initialize the tool with your API key
parallel_tool = ParallelWebSystemsToolSpec(
    api_key="your-api-key-here",
)

# Create an agent with the tool
agent = FunctionAgent(
    tools=parallel_tool.to_tool_list(),
    llm=OpenAI(model="gpt-4o"),
)

# Use the agent to perform web research
response = await agent.run("What was the GDP of France in 2023?")
print(response)

Get your API key from the Parallel AI Platform to authenticate requests. The tool provides two main functions: search for web research with structured excerpts, and extract for converting URLs to clean content.

Source README

Parallel AI Tool

This tool provides integration between LlamaIndex and Parallel AI's Search and Extract APIs, enabling LLM agents to perform web research and content extraction.

  • Search API: Returns structured, compressed excerpts from web search results optimized for LLM consumption
  • Extract API: Converts public URLs into clean, LLM-optimized markdown including JavaScript-heavy pages and PDFs

Installation

pip install llama-index-tools-parallel-web-systems

Setup

  1. Get your API key from Parallel AI Platform
  2. Set your API key as an environment variable or pass it directly

Usage

from llama_index.tools.parallel_web_systems import ParallelWebSystemsToolSpec
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.llms.openai import OpenAI

### Initialize the tool with your API key
parallel_tool = ParallelWebSystemsToolSpec(
    api_key="your-api-key-here",
)

### Create an agent with the tool
agent = FunctionAgent(
    tools=parallel_tool.to_tool_list(),
    llm=OpenAI(model="gpt-4o"),
)

### Use the agent to perform web research
response = await agent.run("What was the GDP of France in 2023?")
print(response)

Available Functions

search

Search the web using Parallel AI's Search API. Returns structured excerpts optimized for LLM consumption.

Parameters:

  • objective (str, optional): Natural-language description of what to search for
  • search_queries (list[str], optional): Traditional keyword search queries (max 5)
  • max_results (int): Maximum results to return, 1-40 (default: 10)
  • mode (str, optional): 'one-shot' for comprehensive results, 'agentic' for token-efficient results
  • excerpts (dict, optional): Excerpt settings, e.g., {'max_chars_per_result': 1500}
  • source_policy (dict, optional): Domain and date preferences
  • fetch_policy (dict, optional): Cache vs live content policy

At least one of objective or search_queries must be provided.

Example:

from llama_index.tools.parallel_web_systems import ParallelWebSystemsToolSpec

parallel_tool = ParallelWebSystemsToolSpec(api_key="your-api-key")

### Search with an objective
results = parallel_tool.search(
    objective="What are the latest developments in renewable energy?",
    max_results=5,
    mode="one-shot",
)

for doc in results:
    print(f"Title: {doc.metadata.get('title')}")
    print(f"URL: {doc.metadata.get('url')}")
    print(f"Excerpts: {doc.text[:300]}...")
    print("---")

### Search with specific queries
results = parallel_tool.search(
    search_queries=["solar power 2024", "wind energy statistics"],
    max_results=8,
    mode="agentic",
)

extract

Extract clean, structured content from web pages using Parallel AI's Extract API.

Parameters:

  • urls (list[str]): List of URLs to extract content from
  • objective (str, optional): Natural language objective to focus extraction
  • search_queries (list[str], optional): Specific keyword queries to focus extraction
  • excerpts (bool | dict): Include excerpts (default: True). Can be dict like {'max_chars_per_result': 2000}
  • full_content (bool | dict): Include full page content (default: False)
  • fetch_policy (dict, optional): Cache vs live content policy

Example:

from llama_index.tools.parallel_web_systems import ParallelWebSystemsToolSpec

parallel_tool = ParallelWebSystemsToolSpec(api_key="your-api-key")

#### Extract content focused on a specific objective
results = parallel_tool.extract(
    urls=["https://en.wikipedia.org/wiki/Artificial_intelligence"],
    objective="What are the main applications and ethical concerns of AI?",
    excerpts={"max_chars_per_result": 2000},
)

for doc in results:
    print(f"Title: {doc.metadata.get('title')}")
    print(f"Content: {doc.text[:500]}...")

#### Extract full content from multiple URLs
results = parallel_tool.extract(
    urls=[
        "https://example.com/

FAQ

Common questions

Discussion

Questions & comments · 0

Sign In Sign in to leave a comment.