Optimize GitLab CI/CD Pipelines

This skill creates efficient, scalable, maintainable GitLab CI/CD pipeline configurations across GitLab's full YAML feature set. A pipeline foundation defines clear stages (validate, build, test, security, deploy, cleanup), shared variables, and defaults (a base image, a common before_script, and retry rules for runner failures or timeouts). Advanced job patterns include a Docker-in-Docker build with BuildKit and registry layer caching, tagged by commit SHA and latest, gated by rules for the default branch or merge requests touching specific paths, and matrix testing across multiple language versions and test suites in parallel with coverage extraction and Cobertura reporting.

Use it when building a GitLab pipeline that needs multi-environment deployment, dynamic child pipelines, or matrix testing - not a single linear build-and-test script. It is not meant to skip production safeguards: production deploys should stay manual and gated on a semver tag with allow_failure: false, per the pipeline's own environment-specific deployment pattern.

Given a project's build and deploy requirements, it produces dynamic pipeline generation (includes pulling in local job definitions plus GitLab's SAST and Dependency-Scanning security templates, a job that generates a child pipeline YAML via a Python script, and a trigger job that runs that generated pipeline with a strategy: depend dependency), environment-specific deployment jobs built from a shared YAML anchor (kubectl apply, rollout-status verification, and post-deploy pod listing) - a staging deploy running automatically on the default branch with 2 replicas, and a production deploy requiring manual approval, gated on a semantic-version tag, running with 5 replicas and disallowed failures - and security and compliance jobs (a secrets-detection job excluding test paths, and a license-check job using license_finder with failures disallowed).

stages:
  - validate
  - build
  - test
  - security
  - deploy
  - cleanup

variables:
  DOCKER_DRIVER: overlay2
  DOCKER_TLS_CERTDIR: "/certs"
  BUILDKIT_PROGRESS: plain

default:
  image: alpine:latest
  before_script:
    - apk add --no-cache git curl
  retry:
    max: 2
    when:
      - runner_system_failure
      - stuck_or_timeout_failure

Optimization layers on top include multi-level caching (a yarn.lock-keyed cache for node_modules and the Yarn cache, plus a commit-ref-slug-keyed cache for build output), Kubernetes resource requests and limits tuned per job, and troubleshooting practices - CI_DEBUG_TRACE for detailed job debugging, artifact collection on failed jobs, timeout tuning for long-running steps, strategic use of needs to shorten pipeline execution time, and after_script sections for consistent error handling.

Platform and release engineers maintaining GitLab pipelines who need the full operational pattern set - regular base-image and dependency updates, scheduled maintenance pipelines, pipeline versioning through includes, resource-usage monitoring, merge-request pipelines to validate changes before merge, and proper secret management via GitLab CI/CD variables rather than hardcoded credentials.