Scrape LinkedIn, GitHub, Twitter & YouTube via DeepAPI

Wraps DeepAPI's scraping, research, email, image-generation, and web-search endpoints behind a single consistent request pattern: authenticate with a bearer key, send a unique idempotency key on every POST, set an explicit spend cap on cost-bearing routes, and poll a running job until it succeeds or fails.

Use it when the task needs a supported DeepAPI scraping, research, or email endpoint, and the user has already provided or confirmed the required DeepAPI credentials and scope - specifically when the user asks to scrape public web data or draft, read, or send email through DeepAPI. The installed skill is pinned to a specific version; if a request or API response reports a different skillVersion than the one pinned in this file's own frontmatter, the mismatch gets reported and the run stops rather than self-updating the file, since updates only arrive through the reviewed repository release process with explicit user approval. DEEPAPI_API_BASE_URL and DEEPAPI_API_KEY must both already be set in the environment; if either is missing, the correct move is to stop and ask the user for setup rather than guessing, and the key itself must never be committed, printed, logged, pasted, or otherwise exposed.

Every request follows the same shape: a bearer Authorization header carrying the API key, a JSON content-type header when sending a body, and a unique Idempotency-Key on every POST so a retried request never double-charges or double-sends. Scrape and paid-generation routes require an explicit spend cap, either maxCostUsd or maxCostMicrousd, which sets the customer's own spend ceiling for that run - the final debit is capped by that number and reported back as debitMicrousd. The execution loop is uniform across routes: pick the narrowest endpoint that matches the task, build the request body from that endpoint's own schema and example, send it with the required headers, and if the response comes back with status running, wait the response's own afterSecs delay and call its own next method and path repeatedly until the status resolves to succeeded or failed; a retryable error waits its own retryAfterSecs before trying again, and an HTTP 402 with an insufficient-credits error code means stopping and asking the user to top up credits before retrying the exact same request under the exact same idempotency key. Every finished request gets reported back with its requestId, status, debitMicrousd, costFinal, and the useful part of the output, never just a bare success or failure. Deep research is a representative single-shot example:

{
  "query": "What changed in EU AI Act compliance timelines for API startups?",
  "context": "We sell API tooling to EU customers.",
  "maxCostUsd": "0.10"
}

Scraping covers a wide surface: a general website crawl returning clean text and markdown per page; LinkedIn profile, company, jobs, posts, and a broader people-search-by-role-location-company-or-school route, which requires a spend cap of at least fifty cents given its higher cost; GitHub profile scraping; X/Twitter search-by-query-or-handle, a single user's profile with optional follower and following lists, and a post's full public reply thread, which requires a spend cap of at least twenty cents; and YouTube transcript extraction, returning both plain text and timed segments with an empty result for videos that have no captions at all, channel stats and recent videos, and keyword search - plus three backward-compatible aliases that map onto the corresponding profile or search route for older callers. Every scrape route shares the same safety posture: never expose the key, always send a fresh idempotency key, always set an explicit spend cap before starting, start with a small result-count cap such as maxItems rather than an unbounded one, and poll the returned path while running rather than blocking synchronously. Email endpoints cover creating a draft or, once explicitly approved, actually sending one; reading messages and listing pending drafts, both free zero-cost reads; and sending a specific already-reviewed draft by its id, which re-checks recipient and content policy against the stored draft at send time so a blocked draft simply stays a draft. Email sending stays in draft mode by default unless the user explicitly approves a real send, never accepts a raw inbox id, only an email-identity id or the workspace default, and blocks attachments, hidden HTML, image-only HTML, URL shorteners, and other high-risk send patterns by policy regardless of approval. Beyond scraping and email, three paid, single-shot routes round out the surface: deep research, which answers a question against current web evidence and should get its actual question in the query field with any relevant background kept separate in a context field; image generation from a text prompt, whose response returns base64 data URLs that should be saved to files rather than printed inline; and web search, which returns ranked title, url, and snippet results with the query capped under 500 characters and each snippet treated as a page summary rather than the full content, opening the actual result URL when more than a summary is needed. A final status-polling route reads or refreshes an in-flight request by its id, restricted to request ids created under the same API key.

Agents and workflows that need to scrape public LinkedIn, GitHub, X/Twitter, YouTube, or general website data, run web search or deep research, generate an image, or draft and send email, all through one consistently-shaped, cost-capped, idempotent API surface rather than a different bespoke integration per data source.