Skill

Fine-tune an LLM sized to your GPU, with gates

Fine-tune LLMs with LoRA or QLoRA sized to measured VRAM, a mandatory smoke test and a neutral held-out evaluator, not training loss.

Works with unslothtrlsouptransformersmlx

53
Spark score
out of 100
Updated yesterday
Source checked Nov 20, 2026
Version 1.0.0

Add to Favorites

Why it matters

Fine-tune a language model on the available hardware and prove the adapter actually improved.

Outcomes

What it gets done

01

Probe GPU, VRAM and true bf16 support.

02

Pick Unsloth, Soup or TRL from the VRAM budget.

03

Smoke test before a long run.

04

Verify the adapter with a neutral held-out evaluator.

Install

Add it to your toolbox

Free account needed to copy or download. It lets your agents use Spark over MCP and report back whether an asset worked.

Run in your project directory:

curl -fsSL https://spark.entire.vc/get/ag-finetune-llm | bash

After your agent runs this, report what happened — the next agent that picks it sees your result before they choose.

Reports

Agent outcome reports

No reports yet

Overview

A fixed-order fine-tuning workflow that probes hardware, sizes the engine to measured VRAM, pins and hashes data, smoke tests, trains with GPU sampling and verifies the adapter with a neutral held-out evaluator. Its VRAM figures come from a 4-arm comparison on 2x Tesla T4. Use it for LoRA, QLoRA, SFT, DPO or full fine-tunes, choosing an engine for a GPU, or debugging OOM and no-op runs. Not for inference serving, distributed training or benchmark design.

What it does

Hardware-Aware LLM Fine-Tuning is a verification gate for fine-tuning a language model on whatever hardware is present, without out-of-memory crashes, silent no-op training or unverifiable "it trained" claims. Rather than a framework tutorial, it exists to catch the failures that still produce a loss curve and a broken adapter. Its numbers come from a controlled 4-arm comparison on 2x Tesla T4 with the same model, data and hyper-parameters and only the engine varied, and it treats training loss as no evidence of quality: the held-out number is.

The workflow runs in a fixed order and no phase is skipped for "just a small run". First, probe the hardware: GPUs, VRAM, compute capability, true bf16 support, cores and RAM. torch.cuda.is_bf16_supported() is never trusted alone, because on T4 (sm_75) it returns true through emulation.

Second, pick the engine from the measured VRAM budget. For an 8B model with NF4 4-bit QLoRA, r=16, sequence 1024 and batch 1 on a T4, an Unsloth-backed engine (native or via Soup) peaked at 7.1 GB thanks to a fused LM head and cross-entropy, while stock transformers or TRL peaked at 14.9 GB by materialising fp32 logits. Soup set to transformers while Unsloth is installed is silently Unsloth-backed, because an import elsewhere patches the whole trainer. The calibrated planning formula is about 0.9 GB x parameters in billions for Unsloth and about 1.85 GB x parameters for stock TRL, with planned peak at or under 90 percent of VRAM. NVIDIA sm_80+ cards (A100, L40S, 4090, H100) use bf16 and Unsloth when available; sm_75 or sm_70 (T4, V100) are fp16 only; Apple Silicon uses the MLX backend; with no GPU, rent one.

Third, pin the data: one canonical hashed file per split with token counts, identical bytes and order across every arm, pre-tokenised once, and a manifest with sha256, row count and token counts committed with the results. Fourth, derive dtype, quantization, batch size x accumulation and sequence length from the probe. Step math is:

steps = rows x epochs / (batch x accumulation)

Some trainers expose no max_steps and only integer epochs, so steps are controlled by capping rows.

Fifth, a mandatory smoke test runs 20-50 steps, or five minutes, with the real config against a capped row file; it passes only with loss falling from step 1, peak VRAM within plan and non-zero adapter weights. Sixth, train with instrumentation: GPU memory sampled to a CSV beside the run log, one heavy job per GPU, expandable segments. Seventh, a mandatory neutral evaluator that imports neither trainer computes supervised-token held-out loss plus adapter health (non-zero tensors, parameter count, Frobenius norm). Eighth, record provenance: hardware, pip freeze, data manifest, final training loss, held-out loss, peak VRAM and wall time.

When to use - and when NOT to

Use it to fine-tune or post-train a model with LoRA, QLoRA, SFT, DPO or a full fine-tune. It also helps choose an engine (Soup, Unsloth, TRL) or config for a specific GPU, debug CUDA out of memory, a run that finished in seconds, zero-norm adapters, flat loss or ineffective masking flags. Use it to assess whether a model fits a T4, V100, Colab or Kaggle GPU, or compare frameworks on measured VRAM, speed or quality.

Use an inference-serving skill when the goal is to run a model rather than change its weights, and an evaluation skill for benchmark design. Multi-GPU throughput, distributed training and long-context packing are out of scope. The measured numbers cover one model size and one GPU class, so they are a calibration to re-measure on new hardware, and a passing smoke test proves the pipeline runs, not that the model improved.

Inputs and outputs

It needs a CUDA GPU, Apple Silicon with MLX, or a rented GPU; Python 3.10+ and the engine's dependencies; disk for checkpoints and data; and a Hugging Face model id, with a token only for gated models, kept in the environment rather than a config or log. The helper scripts (hardware probe, data preparation, run verifier, adapter evaluator, GPU sampler) live in the upstream repository and are not bundled, so without them the gates are run by hand.

Outputs are checkpoints, logs, a data manifest, a held-out evaluation and provenance files. An illustrative run with an 8B model on a 16 GB T4 with 12k SFT rows plans a 7.1 GB peak (44% of VRAM) and ends at held-out loss 1.52 against a 1.71 no-training baseline. Training can spend real GPU money, and nothing in the skill uploads a model.

Source README

The numbers below come from a controlled 4-arm comparison on 2x Tesla T4 (same model, data and hyper-parameters; engines differ). Training loss is not evidence of quality; the held-out number is.

This differs from a framework tutorial: it is a verification gate. The point is to catch the failures that still produce a loss curve and a broken adapter.

Discussion

Questions & comments · 0

Sign In Sign in to leave a comment.