How do I set up Trackio to log metrics during ML training?

Call `trackio.init(project=..., space_id=...)` at the start of training, use `trackio.log({...})` to record metric dictionaries per step (or use TRL's built-in `report_to="trackio"` integration), and call `trackio.finish()` to close out the run. The `space_id` parameter syncs metrics to a Hugging Face Space dashboard for remote/cloud training.

What are Trackio alerts and how do I use them?

Alerts are structured, queryable events fired with `trackio.alert(title=..., level=trackio.AlertLevel.WARN)` to flag important training conditions like loss spikes, NaN gradients, or training stalls. They have three severity levels (INFO, WARN, ERROR), are printed to the terminal, stored in the database, shown on the dashboard, and can be forwarded to Slack/Discord webhooks.

How do I retrieve metrics and alerts after a training run?

Use the `trackio` CLI: `trackio list projects/runs/metrics` to discover available data, `trackio get project/run/metric` to retrieve summaries and values, and `trackio list alerts --project --json` to retrieve alerts. Add `--json` for programmatic output suited for automation and LLM agents.

Can I use Trackio for autonomous ML experimentation?

Yes. The recommended workflow is to set up training with `trackio.alert()` calls for diagnostic conditions, launch training in the background, poll for new alerts with `trackio list alerts --project --json --since `, inspect metric values, then iterate by adjusting hyperparameters and relaunching based on the alerts and metrics shown.

Does Trackio work with Hugging Face's TRL library?

Yes, Trackio integrates with TRL's built-in `report_to="trackio"` parameter, allowing you to log metrics without explicitly calling `trackio.log()` for each step.

Skill

Track ML Training Experiments and Alerts

Name: Trackio - Experiment Tracking for ML Training
Availability: OnlineOnly
Author: Antigravity

Trackio experiment tracking: log ML training metrics, fire diagnostic alerts, and retrieve results via CLI.

Get skill

Works with hugging faceslackdiscord

Antigravity

Own this? Claim it

Spark score

out of 100

Updated 2 months ago

Version 1.0.0

Add to Favorites

Why it matters

Automate the tracking and monitoring of machine learning training runs. Log metrics, fire diagnostic alerts, and visualize progress in real-time via Hugging Face Spaces.

Outcomes

What it gets done

Log training metrics using a Python API.

Define and trigger alerts for training diagnostics (INFO, WARN, ERROR).

Retrieve metrics and alerts via a command-line interface.

Sync experiment data to Hugging Face Spaces for remote monitoring.

Install

Add it to your toolbox

Run in your project directory:

curl -fsSL https://spark.entire.vc/get/ag-hugging-face-trackio | bash

Overview

Trackio - Experiment Tracking for ML Training

Trackio is an experiment-tracking library for logging ML training metrics to a Hugging Face Space dashboard, firing structured diagnostic alerts, and retrieving results via CLI. Use it when logging ML training metrics, firing diagnostic alerts during training, or retrieving results programmatically for autonomous experiment iteration.

What it does

Trackio is an experiment-tracking library for logging and visualizing ML training metrics, syncing to a Hugging Face Space for a real-time monitoring dashboard. It has three interfaces: a Python API for logging metrics (trackio.init(), trackio.log(), trackio.finish(), or TRL's report_to="trackio", passing space_id for remote or cloud training so metrics persist after the instance terminates), a Python API for firing alerts (trackio.alert(title=..., level=trackio.AlertLevel.WARN) at INFO, WARN, or ERROR severity, printed to terminal, stored in the database, shown on the dashboard, and optionally sent to Slack or Discord webhooks), and a CLI for retrieving metrics and alerts after or during training (trackio list, trackio get, trackio show for the dashboard, trackio sync to push to a Space, with --json for programmatic output).

import trackio

trackio.init(project="my-project", space_id="username/trackio")
trackio.log({"loss": 0.1, "accuracy": 0.9})

For autonomous LLM-agent workflows, alerts are the primary mechanism for unattended iteration: an agent inserts trackio.alert() calls for diagnostic conditions like loss spikes, NaN gradients, or training stalls, watches them print to terminal for foreground runs, or polls trackio list alerts --project <name> --json --since <timestamp> for background or detached runs, then reads specific metric values via trackio get and decides whether to stop the run, adjust hyperparameters, and relaunch.

When to use - and when NOT to

Use it when logging ML training metrics for visualization, firing structured diagnostic alerts during training, or retrieving metrics and alerts programmatically - especially for autonomous agent-driven experiment iteration.

Inputs and outputs

Input: training-loop metric values and diagnostic conditions to log or alert on. Output: metrics synced to a Hugging Face Space dashboard, alerts surfaced to terminal, database, dashboard, or webhook, and CLI-queryable JSON summaries of both.

Integrations

Syncs to Hugging Face Spaces for dashboards, integrates with TRL via report_to="trackio", and sends alert notifications to Slack or Discord webhooks.

Who it's for

ML engineers and autonomous agents running training experiments who need real-time metric visualization and structured, pollable alerts instead of scanning raw logs.

Its recommended autonomous workflow runs in five steps: set up training with alerts for known diagnostic conditions, launch the run in the background, poll for new alerts since a given timestamp, read specific metric values to confirm what triggered, and iterate - stopping the run, adjusting hyperparameters, and launching a fresh one based on exactly what the alerts and metrics showed at that point in the run. Its dashboard-oriented commands - trackio show and trackio sync - complement the query commands, launching a local view or pushing the run's data to its Hugging Face Space respectively.

FAQ

Common questions

Discussion