Configure Prometheus for Monitoring

Prometheus Configuration guides metric collection, scrape jobs, service discovery, recording rules and alerts for applications and infrastructure. It connects instrumented metrics endpoints to a Prometheus server, with Alertmanager for notifications, Grafana for visualization and Thanos or Cortex as long-term storage options.

The configuration patterns cover global scrape and evaluation intervals, external labels, alert routing targets and rule-file loading. Scrape examples include the server itself, node exporters, annotated Kubernetes pods and applications using TLS client credentials. Relabeling can derive instance labels, preserve namespace and pod identity and select annotation-enabled targets.

Use it to establish monitoring, add metric targets, implement discovery or prepare recording and alert rules. Clarify collection goals, infrastructure constraints and required inputs before adopting a deployment example. It is scoped to Prometheus configuration rather than unrelated monitoring products or general application work.

Pin and inspect chart or image versions and installation values. Keep monitoring APIs private and review labels and configuration responses for tenant data or secrets. Bound label cardinality so instrumentation does not create uncontrolled time-series growth. Example alert thresholds are illustrative, not universal service-level objectives.

Inputs include reachable metrics endpoints, target labels, discovery credentials, TLS files where needed, retention needs, storage capacity and routing destinations. Outputs are a reviewed server configuration, target definitions, recording and alert rules and validation evidence. A running server with no valid targets or usable rules is not sufficient evidence of completed monitoring.

The Kubernetes Helm pattern needs a suitable StorageClass and capacity in addition to the persistent-volume fragment. Inspect rendered manifests before installation. The loopback-bound Compose pattern supplies a server and data volume, but omits Alertmanager, exporters, Kubernetes credentials and RBAC, TLS material and mounts for the full rule configuration. Supply those dependencies deliberately or remove unused jobs rather than expecting the fragment to create them.

A complete configuration template, deeper scrape and recording references and the validation helper are external prerequisites, not bundled files. Obtain the original material or an existing installation, resolve paths and inspect its contents and applicable licenses. Do not fabricate a replacement helper or claim an unavailable reference was followed. Inline steps can proceed independently within the approved scope.

Static discovery defines fixed endpoints and labels. File discovery refreshes target lists from JSON or YAML, while Kubernetes discovery selects services and pods through metadata. The examples use annotations to select scrape eligibility, scheme, path and port. Actual API access and authorization must be configured for the deployment.

Recording rules precompute frequently queried expressions such as request rate, error percentage, latency quantiles and resource utilization. An absent error series may need an explicit zero-fill strategy, and zero traffic must not become an assumed healthy percentage. Alert rules add sustained-condition windows, severity labels and explanatory annotations for availability, errors, latency, CPU, memory and disk.

Use promtool to validate configuration and rules before relying on them. One exact configuration check is:

promtool check config prometheus.yml

Troubleshooting inspects scrape targets, active configuration and query results; assert their content rather than treating a responding API as proof of collection. Operational guidance includes consistent metric naming, appropriate intervals, capacity-based retention, self-monitoring, multiple instances for availability and federation for larger deployments. Document custom metrics so the resulting signals can be interpreted.

This skill suits platform and application maintainers who can supply endpoints, discovery access, storage and notification infrastructure. It provides setup patterns and checks, without claiming that omitted components are installed or that the example thresholds establish a production monitoring policy.