Train Language Models on Hugging Face Jobs

TRL Training on Hugging Face Jobs trains and fine-tunes language models with TRL (Transformer Reinforcement Learning) on fully managed Hugging Face cloud GPU infrastructure - no local GPU setup, with results automatically saved to the Hub. TRL supports SFT (standard instruction tuning), DPO (preference-based alignment), GRPO (online RL), and reward modeling; hf_doc_search/hf_doc_fetch pull the latest method-specific docs on demand. Unsloth is recommended instead of standard TRL when GPU memory is limited (60% less VRAM), speed matters (2x faster), models exceed 13B parameters, or training Vision-Language Models (via FastVisionModel).

Key directives: always submit training jobs via the hf_jobs() MCP tool (hf_jobs("uv", {...})) with the script passed inline as a string, not saved to a local file unless explicitly requested, and not run via bash trl-jobs commands; every training script should include Trackio for real-time monitoring; and after submission, report the job ID, monitoring URL, and estimated time rather than polling automatically.

Before any job: confirm a paid HF plan (Pro/Team/Enterprise, required for Jobs) and authentication via hf_whoami(); critically, pass secrets={"HF_TOKEN": "$HF_TOKEN"} in the job config, since the training environment is ephemeral and unsaved results are lost entirely without a Hub push; confirm the dataset exists on the Hub or is loadable and matches the training method's expected format (SFT: messages/text/prompt-completion; DPO: chosen/rejected; GRPO: prompt-only), validating unknown datasets before GPU training; and set a timeout well above the 30-minute default, since a timeout kills the job and loses all progress - 1-2 hours minimum is recommended.

Because jobs are asynchronous and can run for hours, the workflow is: build the training script with Trackio included, submit immediately via hf_jobs(), report the job ID/monitoring URL/Trackio dashboard/estimated time, then wait for the user to request status checks rather than polling. SFTConfig/DPOConfig use max_length (not max_seq_length) to control tokenized sequence length, defaulting to 1024 tokens truncated from the right - overridden higher for longer context, lower for memory constraints, or None for vision models to avoid cutting image tokens.

Four submission approaches are covered, in order of preference for Claude Code: UV scripts (the default choice - PEP 723 inline-dependency Python passed directly to hf_jobs("uv", {...}), with local file paths explicitly unsupported since jobs run in isolated Docker containers - scripts must be inline code, or a public/Hub/GitHub/Gist URL); TRL's own maintained example scripts run by URL with script_args; discovering community UV scripts published as Hub datasets under the uv-scripts organization; and the hf jobs CLI directly when the MCP tool isn't available, with strict syntax rules (hf jobs uv run, all flags before the script URL, --secrets plural) plus ps/logs/inspect/cancel subcommands for status. A fourth, lower-priority option is the trl-jobs package for one-liner terminal training outside the Claude Code context.

Hardware selection scales with model size: t4-small for under 1B params (demos only, skip eval steps), t4-medium/l4x1 for 1-3B (development), a10g-small/large for 3-7B (production), a10g-large/a100-large for 7-13B (LoRA recommended), and a100-large/a10g-largex2 for 13B+ (LoRA recommended); multi-GPU is handled automatically by TRL/Accelerate. Timeout guidelines scale similarly (10-30 min for a quick demo, 1-2 hours for development, 4-6 hours for 3-7B production training) with a 20-30% buffer recommended for loading, checkpointing, and Hub push overhead; a cost/time estimator script is available and should be offered whenever a job's parameters are known and the run will likely exceed an hour or $5.

Dataset validation is treated as mandatory for unknown or custom datasets (and critical for DPO, where ~90% of datasets need column-name mapping) since roughly half of training failures are dataset-format mismatches that a ~$0.01, under-a-minute CPU validation run catches before an expensive GPU job fails - the validator reports READY, NEEDS MAPPING (with copy-paste mapping code), or INCOMPATIBLE per training method. After training, models can be converted to GGUF format for local inference with llama.cpp, Ollama, LM Studio, Jan, or GPT4All, with quantization (4/5/8-bit) shrinking a 7B model from ~14GB to 2-8GB.

Common failure modes and fixes: out-of-memory (reduce per_device_train_batch_size and raise gradient_accumulation_steps to keep the effective batch size near 128, enable gradient_checkpointing, or upgrade hardware); misformatted datasets (run the dataset inspector and apply its suggested mapping); job timeout (check actual runtime in logs, increase timeout with buffer, or reduce scope via fewer epochs/max_steps, and enable periodic checkpoint saving with hub_strategy="every_save"); Hub push failures (verify secrets, push_to_hub, hub_model_id, auth, and write permissions); and missing dependencies (add them to the script's PEP 723 header).

Use this skill when fine-tuning language models on cloud GPUs without local infrastructure, training with TRL methods (SFT, DPO, GRPO), running training jobs on Hugging Face Jobs infrastructure, converting trained models to GGUF for local deployment, ensuring trained models are permanently saved to the Hub, or using modern optimized-default training workflows. Do not treat the output as a substitute for environment-specific validation, testing, or expert review, and stop to ask for clarification if required inputs, permissions, safety boundaries, or success criteria are missing.

ML engineers fine-tuning language models via TRL on Hugging Face's managed cloud GPU infrastructure who need correct job submission syntax, dataset validation, Hub-saving safeguards, and hardware/timeout sizing rather than trial-and-error against an ephemeral training environment.