Train Vision Models on Hugging Face Jobs

Trains object detection (D-FINE, RT-DETR v2, DETR, YOLOS), image classification (any timm/ model or Transformers classifier), and SAM/SAM2 segmentation models on Hugging Face Jobs' managed cloud GPUs, with results automatically pushed to the Hub. It runs production-ready HfArgumentParser-based training scripts (object_detection_training.py, image_classification_training.py, sam_segmentation_training.py), each with dataset requirements enforced before submission: object detection needs an objects column with bbox/category (auto-converts xywh/xyxy, auto-remaps string categories to integers, auto-generates a missing image_id); image classification needs image and label columns; SAM/SAM2 needs image and mask columns plus a bbox or point prompt in a prompt column or dedicated bbox/point column. A bundled dataset_inspector.py validates format before spending GPU time - the skill flags format mismatches as the number-one cause of training failures and says to always run it on any dataset not already known-good.

job_info = api.run_uv_job(
    script="path/to/training_script.py",
    script_args=["--dataset_name", "cppe-5"],
    flavor="a10g-large",
    timeout=14400,
    secrets={"HF_TOKEN": get_token()},
)

Use it for fine-tuning a bounding-box detector, image classifier, or segmentation model on a custom dataset on Hugging Face Jobs, or locally. It requires a paid HF plan (Pro, Team, or Enterprise), since Jobs are gated behind one, plus a write-scoped token passed through job secrets, not implicit token resolution, which the skill calls unreliable in Jobs. It draws a sharp boundary against two sibling skills: general Jobs mechanics (hardware flavors, cost estimation, secrets syntax, scheduled jobs) belong to hugging-face-jobs, and TRL-based language-model fine-tuning (SFT/DPO/GRPO) belongs to hugging-face-model-trainer - this skill is vision-specific only. Its critical-directives list catches the failure modes that actually happen in production: hf_jobs() is an MCP tool, not an importable Python function, and the fallback HfApi().run_uv_job() has different syntax for the script parameter, token secret, and timeout units; the training script must inject HF_TOKEN into training_args.hub_token before the Trainer is constructed, or the Hub push returns 401 even with push_to_hub=True set; Optional[bool] HfArgumentParser fields like greater_is_better need an explicit True/False value, while plain bool fields are bare flags; and the default 30-minute job timeout is far too short for object detection, needing 2-12+ hours depending on dataset size.

Input is a Hub dataset ID plus a model checkpoint - recommended starting points are ustc-community/dfine-small-coco for detection, timm/mobilenetv3_small_100.lamb_in1k for classification, and facebook/sam2.1-hiera-small for segmentation, all under 100M params and trainable on a t4-small, $0.40/hr GPU. Output is a fine-tuned model automatically pushed to a Hub repo (hub_model_id) plus, for object detection and SAM, an automatic Trackio dashboard at https://huggingface.co/spaces/{username}/trackio (image classification needs --report_to trackio explicitly). The five-step submission workflow: verify prerequisites, run the dataset inspector, ask the user about a quick-test subset, validation split, and GPU flavor via AskUserQuestion, prepare the correct training script with its required flags, then save the script, submit the job, and report the job ID plus monitoring links without auto-polling. A documented failure-mode table covers CUDA OOM, dataset format errors, Hub-push 401s, job timeouts, and a single-class mAP tensor-shape bug already patched in the template script.

Built on Hugging Face Jobs (managed cloud GPU execution via hf_jobs/run_uv_job), the Transformers Trainer API, timm for classification backbones, and Trackio for training dashboards. It cross-references hugging-face-jobs for general Jobs infrastructure questions and hugging-face-model-trainer for language-model fine-tuning, and ships companion scripts (dataset_inspector.py, estimate_cost.py) plus deep-dive reference docs for timm integration, SAM2 fine-tuning, and Hub-persistence verification.

ML practitioners fine-tuning a vision model - detection, classification, or segmentation - on a custom Hub dataset who want a managed-GPU workflow with pre-validated dataset formats and pre-solved Jobs and Hub-authentication gotchas, rather than assembling a Trainer script and cloud-GPU submission from scratch.