Train Vision Models on Hugging Face Jobs

Trains three types of vision models on managed cloud GPUs via Hugging Face Jobs, with results automatically pushed to the Hub: object detection (D-FINE, RT-DETR v2, DETR, YOLOS), image classification (any timm or Transformers classifier, such as MobileNetV3, MobileViT, ResNet, or ViT/DINOv3), and SAM/SAM2 segmentation using bounding-box or point prompts. It documents dataset requirements per task - object detection needs an objects column with bbox/category sub-fields, classification needs image/label columns, SAM/SAM2 needs image/mask plus a bbox or point prompt - and a mandatory dataset-validation step through dataset_inspector.py that must run before spending GPU time. It lays out a five-step training workflow: verify prerequisites, validate the dataset, ask the user about dataset size and validation split via the AskUserQuestion tool, prepare the training script, then submit and report. It also gives per-task recommended-model tables with parameter counts and hardware guidance (t4-small at 16GB VRAM handles all recommended object-detection and classification models, which are under 100M params), required HfArgumentParser training flags per task, timeout guidance scaled to dataset size, Trackio dashboard monitoring, and six named failure modes with fixes: CUDA out-of-memory, dataset format errors, Hub push 401s, job timeout, a missing test split, and a single-class tensor error.

Use it when fine-tuning an object detection, image classification, or SAM/SAM2 segmentation model on a custom dataset and needing the result permanently saved to the Hugging Face Hub. For text or language model fine-tuning (SFT, DPO, GRPO), it defers to a separate hugging-face-model-trainer skill; for non-training-specific Jobs questions like secrets, hardware flavors, token passing, or scheduled jobs, it defers to the general hugging-face-jobs skill.

Input is a Hub dataset ID plus the target vision task; the skill validates the dataset's column format before committing GPU time:

uv run scripts/dataset_inspector.py --dataset username/dataset-name --split train

Output is a submitted Hugging Face Jobs training run - through either the hf_jobs MCP tool or HfApi().run_uv_job() - using one of three production-ready scripts (object_detection_training.py, image_classification_training.py, sam_segmentation_training.py), plus the job ID, a monitoring URL, a Trackio dashboard link, and cost/time estimates.

Built on Hugging Face Jobs (the hf_jobs MCP tool or the huggingface_hub Python API's run_uv_job()), the Transformers Trainer and HfArgumentParser, timm for classification backbones, and Trackio for training dashboards at https://huggingface.co/spaces/{username}/trackio. Requires a paid Hugging Face plan (Pro, Team, or Enterprise) for Jobs access.

ML engineers and coding agents fine-tuning vision models on cloud GPUs without local GPU setup, who need the trained model automatically persisted to the Hub rather than lost when the ephemeral job environment shuts down.