Generate operational runbooks for incidents and deployments

This skill turns an agent into a runbook writer for operational procedures, incident response, system maintenance, and automation tasks. It structures every runbook around the same essential components: purpose and scope, prerequisites (access, tools, conditions), numbered step-by-step procedures, a verification step for each action, a rollback procedure, emergency escalation contacts, and clear success criteria. The standard template adds an overview block (purpose, estimated time, risk level), a pre-execution checklist, execution steps with a command/expected-output pair and a checkbox verification, a troubleshooting table, and a post-execution checklist.

It covers three specialized runbook types with full worked examples: an incident-response runbook for database connection-pool exhaustion (acknowledging a PagerDuty alert, checking pg_stat_activity for active connections and blocking queries, and named escalation triggers like connections not decreasing within 10 minutes); a production deployment runbook (verifying staging health, taking a pg_dump backup, enabling Kubernetes maintenance mode via a kubectl patch on a ConfigMap, and deploying with helm upgrade --wait); and a weekly maintenance runbook for log rotation and Docker image cleanup (logrotate -f, docker system prune --filter until=168h). It also shows how to make a runbook executable by embedding it as a bash health-check script with set -euo pipefail and explicit pass/fail checks.

Use this skill when documenting an operational procedure that a team must execute consistently and safely, especially under incident pressure - an incident-response playbook, a deployment procedure, or a recurring maintenance task. It insists on including a rollback procedure and named risk level for anything it produces, and flags vague instructions ("restart the system" instead of the specific command), missing verification steps, missing rollback procedures, and commands without expected outputs as anti-patterns to avoid. It is a documentation and template skill, not itself an automation or orchestration system - the bash health-check example is illustrative of how to embed automation, not a delivered tool.

Input: the operational procedure to document (incident type, deployment process, or maintenance task) along with its risk level and required access. Output: a complete runbook following the standard template - overview, pre-execution checklist, numbered execution steps with commands and expected outputs, a rollback procedure, a troubleshooting table, and a post-execution checklist - written in active voice with exact commands and file paths.

The worked examples reference PagerDuty's incident API, PostgreSQL's pg_stat_activity system view, Kubernetes (kubectl, ConfigMaps, rollout restarts), Helm chart deployments, pg_dump for database backups, logrotate, and Docker's image/system prune commands.

SREs, DevOps engineers, and on-call teams who need consistent, safe-to-execute runbooks for incidents, deployments, and maintenance - and a quarterly review discipline (update after system changes, validate in disaster-recovery tests) for keeping them accurate.