Automate Browser Tasks with Puppeteer

This skill provides Puppeteer patterns for browser automation, web scraping, end-to-end testing, and performance optimization, built around five principles: reliability first (explicit wait strategies, error handling, retry mechanisms), performance optimization (efficient selectors, minimized page loads, browser caching), maintainability (modular, reusable helper functions), real-world resilience (network delays, dynamic content, varying load times), and security awareness (safe credential handling, respecting robots.txt and rate limits).

It covers browser launch configuration (headless mode, sandbox flags, viewport, slowMo/devtools for debugging), robust wait strategies (waiting for an element to appear and, optionally, stop moving before interacting with it, and waiting for the network to go idle), error handling and retry logic (a generic retry wrapper and a safe-click helper built on it), dynamic content handling (scrolling to trigger infinite-load content, and waiting for a selector to reach an expected element count), form automation (filling and verifying input values, uploading files and waiting for the upload to register), data extraction (a selector-driven extractor that supports single and multiple-element queries with per-key error handling), page performance optimization (blocking image/stylesheet/font requests, enabling the cache, setting a custom user agent), and a page health-check utility that captures title, URL, JavaScript errors, and Puppeteer's own performance metrics.

const waitForNetworkIdle = async (page, timeout = 30000) => {
  await page.waitForLoadState('networkidle', { timeout });
};

Use this skill when building Puppeteer scripts that need to survive real-world conditions - dynamic/lazy-loaded content, flaky selectors, file uploads, or data extraction from unpredictable pages - rather than a quick one-off script with fixed timeouts. It's suited to scraping, end-to-end test automation, and page health checks that need retry logic and explicit waits baked in.

It is not a fit for sites where automation is disallowed or rate-limited - the skill itself calls out respecting robots.txt and rate limiting, and stealth/anti-bot techniques are flagged as something to use only "when necessary", not a default.

Inputs are the Puppeteer page object plus target selectors, form values, or file paths depending on the task. Outputs are the executed automation result: extracted data objects, a health-check report (title, URL, errors, performance metrics), a submitted form, an uploaded file, or a page scrolled to load all lazy content - each helper function returns either the result or throws a descriptive error after retries are exhausted.

Engineers building browser automation, scraping pipelines, or end-to-end test suites with Puppeteer who want production-ready helpers for waiting, retrying, extracting data, and handling forms and file uploads instead of writing that resilience logic from scratch for every script.