Search the web and fetch content with zero-config fallbacks
Pi Agent's zero-config web-access package: search, fetch PDFs/YouTube/GitHub, with hard query-count minimums by request type.
Why it matters
Enable Pi Agents to retrieve current web information, synthesize answers from multiple search queries, and fetch content from URLs, PDFs, YouTube videos, and GitHub repositories without requiring API keys.
Outcomes
What it gets done
Run batched web searches with automatic fallback across Exa, Perplexity, and Gemini providers
Fetch and convert URLs, PDFs, and YouTube transcripts into readable markdown format
Clone GitHub repositories for local file exploration instead of scraping web pages
Execute deep research workflows with 8+ varied queries across multiple batches
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/ag-pi-web-search | bash Overview
Web Search
Gives Pi Agent a zero-config web-access package covering search, code-context lookups, and fetching PDFs, YouTube, and GitHub content to markdown, with a DeepAPI fallback. Use it whenever a Pi task needs current web information or content fetches. Always pass workflow: none, and meet the hard query-count minimum for the phrasing used.
What it does
Gives Pi Agent its own web-access package (pi-web-access, installed globally) for current web information, page fetches, PDFs, YouTube, and GitHub content - zero-config via Exa MCP with no API key required, falling back from Exa to Perplexity to Gemini if needed.
When to use - and when NOT to
Use it when a Pi Agent task needs current web information, page fetches, PDFs, YouTube, or GitHub content, or specifically when Pi should use its own web-access package instead of another agent's browser tool. Every web_search call must include workflow: "none" with no exceptions, for a single query or a batched list of queries, since this skips the interactive browser curator popup the user does not want opening:
web_search({ queries: ["query 1", "query 2"], workflow: "none" })
Inputs and outputs
Four tools are available: web_search returns synthesized answers with citations and can be called many times per turn (always with workflow: "none"); code_search is zero-key Exa code-context search, preferred over generic web_search for library, API, or code lookups; fetch_content fetches one or more URLs to markdown and handles PDFs, YouTube, and GitHub; and get_search_content pulls the full text of big pages (over 30k characters) that get truncated in normal responses, so they don't blow context. fetch_content behaves differently by source: GitHub URLs are cloned rather than scraped, giving real files plus a local path to explore with read/bash (private repos need the gh CLI), which is the right choice for dev work; PDFs are auto-extracted to markdown in ~/Downloads/, readable in sections, text-only with no OCR; and YouTube or video URLs return full raw transcripts plus frame extraction, which needs a GEMINI_API_KEY (not zero-config) and, for frame extraction specifically, ffmpeg and yt-dlp.
Integrations
Query counts are hard minimums keyed to the user's phrasing, and must be counted before answering rather than stopped short: "web search" needs at least 2 varied queries before synthesizing; "extensive web research" needs at least 4 queries across totally different keywords and angles; and "deep research" needs at least 8 queries across totally different keywords and angles, run in 2-3 successive batches that refine angles after each one. A single batched web_search call counts every query inside queries[] toward that total, and a first batch under the minimum means firing another batch before synthesizing. If the Exa-Perplexity-Gemini chain fails, or ranked results with URLs are specifically needed, DeepAPI web search is the fallback:
test -n "$DEEPAPI_API_KEY" || { echo "DEEPAPI_API_KEY is not set"; exit 1; }
curl -s --max-time 60 "https://deepapi.co/v1/search/web" \
-H "Authorization: Bearer $DEEPAPI_API_KEY" -H "Content-Type: application/json" \
-H "Idempotency-Key: $(uuidgen)" \
-d '{"query": "your search terms", "maxResults": 5, "maxCostUsd": "0.05"}'
Results land in .output (title, url, snippet per item), the query must stay under 500 characters, and full details live in the separate deepapi skill.
Who it's for
Pi Agent tasks that need live web information, code-context lookups, or content fetched from PDFs, YouTube, or GitHub, without opening the interactive browser curator popup.
Source README
Web Search
When to Use
- Use when a Pi Agent task needs current web information, page fetches, PDFs, YouTube, or GitHub content.
- Use when Pi should use its own web-access package instead of another agent browser tool.
The pi-web-access package is installed globally. Zero-config via Exa MCP (no API key), with fallback Exa → Perplexity → Gemini.
CRITICAL: always pass workflow: "none"
Every web_search call MUST include workflow: "none". This skips the interactive browser curator popup (the user does not want it opening). No exceptions - single query or batched queries, always set workflow: "none".
web_search({ queries: ["query 1", "query 2"], workflow: "none" })
Tools
web_search- search the web; returns synthesized answers with citations. Can be called many times per turn. Always passworkflow: "none".code_search- zero-key Exa code-context. Use for library/API/code lookups instead of genericweb_search.fetch_content- fetch URL(s) → markdown; handles PDFs, YouTube, GitHub.get_search_content- big pages (>30k chars) are truncated in responses but stored in full; call this to pull the rest on demand so they don't blow context.
fetch_content specifics
- GitHub URLs are cloned, not scraped - you get real files + a local path to explore with
read/bash(private repos need theghCLI). Use this for dev work. - PDFs → auto-extracted to markdown in
~/Downloads/, readable in sections (text-only, no OCR). - YouTube/video → full raw transcripts + frame extraction. Needs a
GEMINI_API_KEY(not zero-config); frame extraction also needsffmpeg/yt-dlp.
Routing - match the user's phrasing
Always use the web_search tool. These counts are HARD MINIMUMS - count your queries before answering and do not stop short:
- "web search" → at least 2 queries, varied keywords/angles, then synthesize.
- "extensive web research" → at least 4 queries, totally different keywords and angles.
- "deep research" → at least 8 queries, totally different keywords and angles, run across 2-3 successive batches (refine angles after each batch), to learn as much as possible about the topic.
A single batched web_search call counts each query in queries[] toward the total. If your first batch is under the minimum, fire another batch before synthesizing.
Fallback / alternative: DeepAPI web search
If the Exa → Perplexity → Gemini chain fails, or you need ranked results with URLs:
test -n "$DEEPAPI_API_KEY" || { echo "DEEPAPI_API_KEY is not set"; exit 1; }
curl -s --max-time 60 "https://deepapi.co/v1/search/web" \
-H "Authorization: Bearer $DEEPAPI_API_KEY" -H "Content-Type: application/json" \
-H "Idempotency-Key: $(uuidgen)" \
-d '{"query": "your search terms", "maxResults": 5, "maxCostUsd": "0.05"}'
Results are in .output (title, url, snippet per item). Query under 500 chars. Full details: deepapi skill.
Limitations
- Adapted from
davidondrej/skills; verify local paths, tools, credentials, and agent features before acting. - For commands, remote access, scheduling, browser automation, or file-changing workflows, get explicit user approval and confirm the target environment first.
Discussion
Questions & comments · 0
Sign In Sign in to leave a comment.