Track ML Training Experiments and Alerts

Provides three interfaces to Trackio, Hugging Face's lightweight, wandb-API-compatible experiment-tracking library: a Python logging API (trackio.init(), trackio.log(), trackio.finish(), plus automatic integration via TRL's report_to="trackio"), a Python alerting API (trackio.alert(title, text, level) with INFO/WARN/ERROR severities, printed to the terminal, stored in a local SQLite database, shown on the dashboard, and optionally pushed to Slack or Discord webhooks), and a CLI for retrieving logged data (trackio list projects/runs/metrics/alerts, trackio get metric/snapshot/run, trackio show for the dashboard, trackio sync to push a local project to a Hugging Face Space). Metrics are stored locally by default; passing space_id to trackio.init() syncs them to a Space instead, which the skill flags as mandatory for remote or cloud training since local storage is lost when the instance terminates.

import trackio

trackio.init(project="my-project", space_id="username/trackio")
trackio.log({"loss": 0.1, "accuracy": 0.9})
trackio.finish()

Use it when logging ML training metrics, firing diagnostic alerts during training, or retrieving and analyzing already-logged metrics and alerts. Its explicit design center is autonomous LLM-agent iteration: alerts are described as the primary mechanism for LLM agents to diagnose runs and iterate autonomously, since an agent watching a training script's stdout sees ERROR or WARN alerts for conditions like loss divergence, vanishing loss, NaN gradients, or a training stall the moment they're printed, with --since <timestamp> CLI polling as the fallback for detached or background runs. Alerts fired outside a directly-controlled training loop, such as via TRL's report_to="trackio", need a TrainerCallback hook rather than an inline trackio.alert() call, since the agent doesn't own the loop in that case.

Inputs are a project name, an optional config dict of hyperparameters, and a stream of trackio.log({...}) metric dictionaries keyed by name; output is queryable run data - metric time series, system metrics like GPU utilization, and alerts - retrievable via the CLI with a --json flag for programmatic or agent consumption. The recommended autonomous workflow: insert diagnostic trackio.alert() calls into the training loop, launch training in the background, poll trackio list alerts --json --since <timestamp> for new alerts, inspect the metrics around a fired alert with trackio get snapshot --around <step> --window <N> --json, then react - stop and adjust hyperparameters on an ERROR, inspect further on a WARN, or just note progress on an INFO - before comparing across runs and launching the next one.

Drop-in wandb-compatible (import trackio as wandb), with first-class TRL and Transformers integration via report_to="trackio" for automatic loss, learning-rate, and eval-metric capture. Dashboards sync to and are hosted on Hugging Face Spaces, embeddable in a webpage via an iframe with query parameters for project and metric filtering, sidebar visibility, and smoothing. Webhook delivery to Slack or Discord is configured globally via trackio.init(webhook_url=...) or the TRACKIO_WEBHOOK_URL environment variable, with a webhook_min_level floor and a per-alert override.

ML engineers, and specifically LLM agents running training experiments autonomously, who want structured, queryable diagnostic signals alongside standard metric logging, retrievable via a JSON-friendly CLI, rather than watching a raw training log or paying for a hosted tracking service.