Create Datadog Monitors

Datadog Monitor Creator Agent builds Datadog monitors across metric, APM, log, and composite types, with threshold configuration - static critical and warning thresholds plus anomaly-based dynamic thresholds - notification routing, and a monitor-maintenance process aimed at minimizing alert fatigue while catching real issues early.

Use it when defining a new Datadog monitor or restructuring alerting - a metric-alert monitor on infrastructure or business metrics, an APM error-rate or latency monitor, a log-based security monitor, or a composite monitor combining several sub-monitors into one alert. It's specific to Datadog's monitor JSON and query syntax - it doesn't cover other observability platforms, and it assumes metrics, logs, or traces are already flowing into Datadog rather than setting up that instrumentation.

Input is the signal to alert on - a metric, log pattern, or trace - and its severity. Output is Datadog monitor JSON: a metric-alert monitor for high CPU usage with critical and warning thresholds, a no_data_timeframe, and a Slack/PagerDuty-tagged runbook message; an APM monitor alerting on an API error-rate spike above a threshold with investigation steps in the message; a log-alert monitor counting failed logins per user over a rolling window and tagging the security team; a composite monitor combining database connection count, query latency, and CPU sub-monitors with an a && (b || c) query; and a reusable message template using {{value}}, {{threshold}}, and {{alert_transition_time}} variables plus conditional recovery text. It also defines a four-tier severity scale from P1 (completely unavailable, immediate response) to P4 (informational, no action).

Targets the Datadog API's monitor types - metric alert, APM, log alert, composite - with notification routing to PagerDuty for critical production alerts, Slack for team channels, email for summaries, and webhooks for ITSM integration. Dynamic thresholds use anomaly detection with bounds based on historical data, typically two to three standard deviations, to account for seasonal weekly and daily traffic patterns:

{
  "name": "High CPU Usage - Production Servers",
  "type": "metric alert",
  "query": "avg(last_10m):avg:system.cpu.user{env:prod} by {host} > 90",
  "options": {
    "thresholds": {
      "critical": 90,
      "warning": 75
    },
    "notify_no_data": true,
    "no_data_timeframe": 20,
    "evaluation_delay": 60
  }
}

SRE and platform teams building or tuning Datadog alerting who want the maintenance discipline built in - monthly false-positive review, quarterly threshold adjustment, post-incident monitor updates, testing in staging before production, and avoiding the common mistakes it flags: alert fatigue, flapping monitors, misconfigured no-data alerts, timezone-blind business-hours checks, and missing recovery notifications.