Research and Summarize Web Content
A Promptfoo example demonstrating Anthropic's web search and web fetch tools with domain filtering and source citations.
0.121.18Add to Favorites
Why it matters
Leverage AI to perform comprehensive web research and distill findings into concise summaries. Automate the process of gathering information from the internet for deeper analysis.
Outcomes
What it gets done
Fetch information from specified URLs.
Conduct web searches for relevant data.
Synthesize research findings into summaries.
Automate complex research tasks.
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/pfoo-web-tools | bash Steps
Steps in the chain
Overview
Web Tools
A Promptfoo example demonstrating Claude's web search and web fetch tools with domain filtering, citations, and usage limits. Use when evaluating Claude's web tool use for research-style tasks where domain restriction and citation transparency matter.
What it does
This is a Promptfoo example demonstrating Anthropic's web search and web fetch tools, covering both basic URL fetching and multi-step research workflows. It exercises web search (finding relevant information across the internet), web fetch (retrieving full content from specific web pages and PDFs), combined multi-step research workflows, domain-filtering security controls restricting fetches to trusted sources, and citations for source attribution. The example's configuration uses the claude-sonnet-4-6 provider with temperature: 0 and max_tokens: 2500, and two tools: web_search_20250305 (named web_search, max_uses: 2) and web_fetch_20250910 (named web_fetch, max_uses: 3, restricted via allowed_domains to docs.anthropic.com, github.com, openai.com, and arxiv.org, with citations: enabled: true and max_content_tokens: 12000 to cap content size).
When to use - and when NOT to
Use this example when evaluating Claude's web search and web fetch tool use for research-style tasks - documentation analysis, competitive research, technical investigation, security assessment, or academic literature review - especially where domain restriction and citation transparency matter. It is initialized as a self-contained Promptfoo example (npx promptfoo@latest init --example anthropic/web-tools), meant to be run and inspected as a template.
Inputs and outputs
Setup requires an ANTHROPIC_API_KEY environment variable (from console.anthropic.com), set in a .env file or the environment directly. The evaluation is run with promptfoo eval from the examples/anthropic/web-tools directory. It tests direct URL fetch and summarization of specific documentation pages, GitHub repository analysis, a multi-step search-then-fetch-then-synthesis research workflow, and investigation of web-tool security considerations.
Integrations
Depends on Promptfoo, an Anthropic API key, and Claude's web search and web fetch tools, configured with usage limits (2 searches, 3 fetches per evaluation) and a domain allowlist to prevent excessive usage or fetching from untrusted sources.
Who it's for
Developers and evaluators who want to test Claude's web search and web fetch tools for research workflows, with domain-restricted, citation-backed retrieval and usage limits configured as guardrails against untrusted sources and excessive API usage.
Source README
anthropic/web-tools (Anthropic Web Tools)
This example demonstrates Anthropic's web search and web fetch tools, showing both basic URL fetching and comprehensive research workflows.
You can run this example with:
npx promptfoo@latest init --example anthropic/web-tools
cd anthropic/web-tools
Features Demonstrated
- Web Search: Find relevant information across the internet
- Web Fetch: Retrieve full content from specific web pages and PDFs
- Combined Workflows: Multi-step research processes
- Security Controls: Domain filtering for trusted sources
- Citations: Source attribution for transparency
Configuration Highlights
### yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
description: 'Web search and fetch tools'
prompts:
- |
{{task}}
providers:
- id: anthropic:messages:claude-sonnet-4-6
config:
temperature: 0
max_tokens: 2500
tools:
- type: web_search_20250305 # Search the web
name: web_search
max_uses: 2
- type: web_fetch_20250910 # Fetch specific URLs
name: web_fetch
max_uses: 3
allowed_domains: # Restrict to trusted domains
- docs.anthropic.com
- github.com
- openai.com
- arxiv.org
citations:
enabled: true # Enable source citations
max_content_tokens: 12000 # Content size limit
Required Environment Variables
This example requires the following environment variables:
ANTHROPIC_API_KEY- Your Anthropic API key from console.anthropic.com
You can set these in a .env file or directly in your environment.
Running the Example
cd examples/anthropic/web-tools
promptfoo eval
What It Tests
- Direct URL Fetch: Retrieve and summarize specific documentation pages
- GitHub Repository Analysis: Analyze code repositories and their purpose
- Research Workflow: Multi-step research with search → fetch → synthesis
- Security Research: Investigation of web tool security considerations
Use Cases
Perfect for:
- Documentation analysis
- Competitive research
- Technical investigation
- Security assessment
- Academic literature review
Security Notes
- Only allows fetching from trusted domains (docs.anthropic.com, github.com, openai.com, arxiv.org)
- Content limited to 12,000 tokens to prevent excessive usage
- Citations enabled for source transparency
- Usage limits prevent API abuse (2 searches, 3 fetches per evaluation)
FAQ
Common questions
Discussion
Questions & comments · 0
Sign In Sign in to leave a comment.