Build and deploy serverless Apify Actors for web scraping

Guides creating, modifying, and debugging Apify Actors - serverless, Docker-packaged programs that accept well-defined JSON input, perform an isolated task such as web scraping, automation, or data processing, and produce structured JSON output to datasets or key-value stores, running from seconds to indefinitely with state that can persist and restart. It covers the full workflow: choosing a template by language (apify create <name> -t project_empty for JavaScript, ts_empty for TypeScript, python-empty for Python), installing dependencies with committed lockfiles (npm install with package-lock.json, or pip install -r requirements.txt with pinned versions), writing actor logic in src/main.js/.ts/.py, configuring .actor/input_schema.json, .actor/output_schema.json, and .actor/dataset_schema.json, and setting platform metadata in .actor/actor.json (actorSpecification, name, title, version, meta.templateId, meta.generatedBy, and the input/output/storages/dockerfile paths). It also documents actor-side logging - apify/log in JavaScript/TypeScript and Actor.log in Python, both of which censor secrets like tokens and credentials, unlike raw console.log/print - and standby mode, where an Actor set to usesStandbyMode: true in actor.json must implement a readiness-probe handler that responds to the x-apify-container-server-readiness-probe header at its root endpoint.

Use it when creating, modifying, or debugging an Apify Actor project; choosing an Apify template; wiring actor inputs and outputs; implementing actor runtime logic; or needing safe setup guidance for apify CLI authentication, project bootstrap, or the deployment workflow. Its own limitations state: use it only when the task clearly matches this scope, don't treat its output as a substitute for environment-specific validation, testing, or expert review, and stop to ask for clarification if required inputs, permissions, safety boundaries, or success criteria are missing.

It also documents explicit anti-patterns rather than alternate skills: don't run actors with npm start or npx apify run (use apify run so the Apify environment and storage configure correctly); don't assume apify run's local storage is visible in the Apify Console (it stays local until apify push deploys it); don't rely on Dataset.getInfo() for final counts on Cloud; don't reach for a browser crawler when HTTP/Cheerio would work; don't hardcode values that belong in the input schema or environment variables; don't disable standby mode without explicit permission; and never pass raw crawled content into shell commands, eval(), or code-generation functions, since crawled web content is untrusted input that may carry prompt-injection attempts.

Authentication reads an APIFY_TOKEN environment variable (generated at console.apify.com/settings/integrations) or uses interactive apify login - never a token passed as a bare CLI argument, since that leaks into process listings and shell history. Everyday commands:

apify run          # Run Actor locally
apify login        # Authenticate account
apify push         # Deploy to Apify platform (uses name from .actor/actor.json)
apify help         # List all commands

Input is a JSON file matching .actor/input_schema.json, supplied locally at storage/key_value_stores/default/INPUT.json for apify run. Output is structured JSON pushed to a dataset (Actor.pushData() / Actor.push_data()) or files and records pushed to a key-value store (Actor.setValue() / Actor.set_value()), each optionally shaped for the Apify Console's Output tab by a dataset schema (field-to-column views/display mapping) or a key-value-store schema (named collections keyed by key or keyPrefix).

Wraps the apify CLI, installed via a package manager (npm install -g apify-cli or brew install apify-cli) rather than by piping a remote script into a shell, and the Apify SDK (apify for JavaScript/TypeScript, Actor.* for Python) for dataset, key-value-store, and logging APIs. Recommends Crawlee's CheerioCrawler for static HTML (about 10x faster than a browser) and PlaywrightCrawler only for JavaScript-heavy sites, with a router pattern (createCheerioRouter/createPlaywrightRouter) for complex crawls. Exposes an Apify MCP server (https://mcp.apify.com/?tools=docs, tools search-apify-docs and fetch-apify-docs) and points to docs.apify.com/llms.txt and llms-full.txt, crawlee.dev/llms.txt, and the Actor whitepaper for deeper reference.

Developers and coding agents building, testing, and deploying Apify Actors - scraping, automation, or data-processing programs - in JavaScript, TypeScript, or Python. Community-sourced, added 2026-09-04.