Minimize tokens and maximize accuracy in web research

A protocol for fetching web content at the minimum depth needed to answer a question, escalating only when a shallower layer fails. It classifies the input first - a GitHub URL, a specific page URL, a topic or query with no URL, a list of URLs, or a direct file link - and routes to a matching protocol. For GitHub, it prefers the API over scraping: fetch repo metadata plus the README first, since the README alone answers roughly 70% of "what does this repo do" questions, and only escalate to fetching the file tree, then at most 1-3 specific relevant files, if the README doesn't answer the question - never fetching an entire repo, and reading only the top of any file over roughly 300 lines. For a specific URL, it skims headings and the first paragraph first, then a targeted anchor section (200-500 tokens), then a full stripped page capped at 2000 tokens only if the targeted extract fails, and reaches for a browser-rendering fallback only when the plain fetch returns empty, garbled, or JS-placeholder content. For a topic query, it sharpens the raw query before searching, adding specificity like version numbers, "tutorial" or "comparison", or a year, and removing filler like "how do I", scans only titles and snippets from up to 3 results, and fetches the top 1-2 selected pages one at a time, stopping as soon as one answers the question - a snippet that already contains the answer means no fetch happens at all. Multiple URLs get a first-pass skim of every one, then a deep fetch of only the 1-3 most relevant, each summarized in 3-5 sentences before combining, since raw content from multiple pages is never dumped together. File links route by type: Markdown, text, or CSV read in full, PDFs go through a text-extraction path only, JSON or YAML get parsed and summarized rather than dumped, and large files get their first 100 and last 20 lines plus a targeted search for relevant sections.

Use it whenever a task involves accessing a URL, a GitHub repo, or running a search query, to prevent wasteful full-page fetching. Explicit anti-patterns it forbids: fetching a full page for a simple fact instead of using the snippet or a targeted anchor; using a browser-rendering fallback on a static site; searching with the raw, unsharpened user query; fetching more than 3 search results; dumping raw HTML into context instead of stripping to Markdown; fetching content "just in case" that isn't needed to answer; re-fetching a URL already fetched in the same turn; and fetching an entire GitHub repository instead of its README plus targeted files. It cannot bypass CAPTCHAs, strict bot protections, or hard paywalls, and unauthenticated GitHub API calls can hit rate limits under heavy use.

Input is a GitHub URL, a specific page URL, a topic or question, a list of URLs, or a direct file link. GitHub metadata and content are read via the API rather than scraped HTML:

GET https://api.github.com/repos/{owner}/{repo}
GET https://api.github.com/repos/{owner}/{repo}/git/trees/{ref}?recursive=1
GET https://api.github.com/repos/{owner}/{repo}/contents/{path}?ref={ref}
GET https://api.github.com/repos/{owner}/{repo}/readme

Fetched pages have navigation, cookie banners, related-article blocks, footers, share buttons, and ads stripped out, keeping only the main body, code blocks, data tables, and numbered procedures. Output is a fixed, attributed structure: a Source line naming the URL or the search query used, a 2-5 sentence Summary, a direct Answer, and a Confidence rating based on source quality, combined across sources when more than one was used - never raw HTML fragments, full page dumps, or unattributed information.

Built around three fetch primitives: a lightweight URL-content reader for static pages, a browser-rendering subagent reserved for JavaScript-rendered or auth-gated pages, and a web-search tool for topic queries, plus direct GitHub REST API calls for repository research instead of scraping GitHub's HTML.

Any agent doing web or repository research that needs to answer a question cheaply and accurately, stopping at the shallowest fetch layer that actually answers it, rather than defaulting to full-page or full-repo fetches that burn tokens on content nobody reads.