Orchestrate Production-Ready ML Pipelines

The skill orchestrates a complete ML pipeline build across four phases and seven specialized subagents, each handling a distinct MLOps responsibility with outputs handed off to the next phase. Phase 1 (Data and Requirements Analysis) runs a data-engineer subagent designing ingestion, schema validation (Pydantic/Great Expectations), data versioning (DVC or lakeFS), storage layering, and a data-scientist subagent designing feature engineering, a feature-store schema (Feast/Tecton), model requirements, and A/B experiment design. Phase 2 (Model Development and Training) runs an ml-engineer subagent building the training pipeline with hyperparameter optimization (Optuna/Ray Tune), distributed training (Horovod/PyTorch DDP), experiment tracking (MLflow/Weights and Biases), and model-registry promotion workflows, followed by a python-pro subagent that productionizes the code with error handling, performance profiling, and a full test suite (unit, integration, model-quality, and performance-regression tests). Phase 3 (Production Deployment and Serving) runs an mlops-engineer subagent building serving infrastructure (FastAPI/TorchServe, KServe/Seldon Core), deployment strategies (blue-green, canary, shadow), CI/CD (GitHub Actions/GitLab CI, ArgoCD GitOps), and infrastructure as code (Terraform, Helm, Docker, Vault), followed by a kubernetes-architect subagent handling workload orchestration (Kubeflow, GPU sharing, spot instances), autoscaling (HPA/VPA/KEDA), Istio service mesh, and storage/caching for the ML platform. Phase 4 (Monitoring and Continuous Improvement) runs an observability-engineer subagent covering model-performance monitoring, statistical drift detection (KS test, PSI), system observability (Prometheus, Grafana, Jaeger/Zipkin, ELK/Loki), PagerDuty/Opsgenie alerting with automated retraining triggers, and cost tracking by model or experiment.

The workflow exposes configuration options for experiment tracking (mlflow, wandb, neptune, clearml), feature store (feast, tecton, databricks, custom), serving platform (kserve, seldon, torchserve, triton), orchestration (kubeflow, airflow, prefect, dagster), cloud provider, deployment mode (realtime, batch, streaming, hybrid), and monitoring stack (prometheus, datadog, newrelic, custom). Success is measured against concrete thresholds: under 0.1% data quality issues in production, sub-second feature serving latency, under 5% model-performance degradation before retraining, 99.9% serving uptime, under 200ms p99 inference latency, automated rollback within 5 minutes, under 1 hour from commit to production, and over 60% spot-instance utilization with under 20% infrastructure waste.

Use it when designing or building a complete, production-grade ML pipeline that needs multiple specialist perspectives (data engineering through observability) coordinated in sequence. It is not a substitute for environment-specific validation, testing, or expert review, and it stops for clarification when required inputs, permissions, safety boundaries, or success criteria are missing.

Input is the ML system's requirements, passed as arguments through each phase's subagent prompts. Output is an end-to-end ML pipeline with automation, documentation and runbooks, production infrastructure as code, a full monitoring and alerting system, CI/CD pipelines, cost optimization, and disaster-recovery/rollback procedures.

Teams building a production ML system who need data engineering, feature/model design, training productionization, deployment infrastructure, and observability handled as one coordinated multi-agent workflow rather than as separate, disconnected efforts.