Build and Deploy LLM Fine-Tuning Pipelines

This skill designs and implements comprehensive fine-tuning pipelines for large language models, covering distributed training, memory optimization, data processing, and deployment. Data processing formats instruction-following datasets into a consistent instruction/input/response prompt template, tokenizes with a configurable max length and a pad-token fallback, and applies quality filters on text length (roughly 50-10,000 characters) and an optional quality score threshold above 0.7. Model setup loads the base model in bfloat16 with automatic device mapping and flash attention enabled, then applies LoRA for parameter-efficient fine-tuning (rank 16, alpha 32, 0.1 dropout, targeting the query, key, value, output, and gate/up/down projection modules), configured through training arguments covering epochs, gradient accumulation, gradient checkpointing, warmup steps, mixed-precision training, periodic evaluation and checkpointing with a save-total limit, and DeepSpeed integration.

Use it when fine-tuning an LLM at a scale that needs distributed training, memory optimization, and production deployment - not a small full-precision fine-tune on a single consumer GPU with no memory constraints. It is not meant to skip evaluation: the pipeline's own best practices require monitoring perplexity, loss curves, and task-specific metrics throughout training, not just watching the loss go down.

Given a base model and an instruction dataset, it produces a DeepSpeed ZeRO Stage 2 configuration (partitioned gradient allgather and reduce-scatter with a 2e8 bucket size, contiguous gradients, no CPU offload, an AdamW optimizer and WarmupLR scheduler with auto-tuned parameters, and bfloat16 enabled), a training monitor (Weights & Biases logging, a perplexity metric computed from shifted cross-entropy loss, and early stopping with a patience of 3), and a production deployment wrapper (compiling the model with torch.compile, enabling the KV cache, and generating responses with temperature-based sampling).

{
    "zero_optimization": {
        "stage": 2,
        "allgather_partitions": true,
        "allgather_bucket_size": 2e8,
        "overlap_comm": true,
        "reduce_scatter": true,
        "reduce_bucket_size": 2e8,
        "contiguous_gradients": true,
        "cpu_offload": false
    },
    "optimizer": {
        "type": "AdamW",
        "params": {
            "lr": "auto",
            "betas": "auto",
            "eps": "auto",
            "weight_decay": "auto"
        }
    },
    "scheduler": {
        "type": "WarmupLR",
        "params": {
            "warmup_min_lr": "auto",
            "warmup_max_lr": "auto",
            "warmup_num_steps": "auto"
        }
    },
    "bf16": {"enabled": "auto"},
    "gradient_clipping": "auto",
    "train_micro_batch_size_per_gpu": "auto"
}

Five memory-optimization techniques stack together: gradient checkpointing trades compute for memory, flash attention roughly halves attention memory use, bfloat16 mixed precision saves about 50% of memory, gradient accumulation simulates larger batch sizes without the memory cost, and LoRA or QLoRA cuts trainable-parameter memory by roughly 90%. Hyperparameter guidance differs by method - learning rates of 1e-5 to 5e-4 for full fine-tuning versus 1e-4 to 2e-3 for LoRA - alongside a small-batch-plus-gradient-accumulation strategy and GPU memory profiling to size batches correctly.

ML engineers fine-tuning LLMs in production who need the full pipeline, not just a training loop - data quality filtering, memory-efficient distributed training via DeepSpeed and LoRA, checkpointed and resumable training, real evaluation metrics, and an inference-optimized deployment wrapper.