Generate Advanced Buildkite CI/CD Pipelines

Buildkite Pipeline Generator helps structure YAML pipelines for CI/CD work. It covers agent targeting, step dependencies, wait steps, artifact transfer, environment scoping, dynamic pipeline uploads, matrix builds, manual approval blocks, plugins, branch conditions, retries, and performance considerations. Examples include build, test, and container steps connected through dependencies and artifacts, as well as generated steps based on changed services.

The guidance emphasizes selecting agent queues for workload needs, using depends_on and wait steps to manage ordering and parallel work, and passing build outputs through artifact_paths and plugins. It also describes matrix configuration for combinations of operating systems and Node versions, block steps for collecting deployment inputs, conditional execution by branch or build state, and automatic retry settings.

Use this skill when creating or refining a Buildkite pipeline that needs clear orchestration across build, test, and deployment work. It is useful when jobs need specialized agent queues, when independent steps can run in parallel, or when generated steps, artifacts, conditions, or approval blocks are required.

Do not copy sample queue names, operating systems, branch rules, image tags, plugin versions, or commands as if they were requirements for a particular project. The YAML is illustrative. Do not expose secrets in pipeline files or logs; Use secret management through environment hooks and plugins, and its Slack example reads a webhook from an environment variable. Confirm plugin availability and configuration in the target Buildkite setup.

Inputs include the repository's build and test commands, service boundaries, agent capabilities, dependency relationships, artifact requirements, branch strategy, environment needs, approval rules, and retry policy. For dynamic pipelines, Examples obtains changed services and builds a set of steps for each service.

The output is a Buildkite pipeline YAML file, or a generator that uploads pipeline steps through buildkite-agent pipeline upload. Pipeline steps may specify labels, commands, agents, environment variables, timeouts, parallelism, artifacts, plugins, depends_on, wait, block, if, branches, retry, and soft_fail. Validate that dependencies and artifact paths match actual commands and repository outputs.

The material names Buildkite agents and pipeline uploads, YAML, Python, Docker, Node.js, JUnit annotations, artifact plugins, and Slack notifications. It includes a sample generator using Python with yaml, json, and os. Examples also refer to build metadata and Buildkite environment variables. These are integration patterns from the source, not proof that any plugin, secret, or queue is configured in a given account.

This skill is for developers and DevOps engineers maintaining Buildkite CI/CD pipelines. Teams can use it to make step sequencing, agent selection, artifact flow, and deployment conditions explicit. Review generated YAML against the actual repository commands, agent queues, security controls, and release process before enabling it.

steps:
  - label: ":hammer: Build"
    command: |
      echo "Building application..."
      make build
    artifact_paths:
      - "dist/**/*"
      - "build-info.json"
    agents:
      queue: "build"