Optimize LightGBM Hyperparameters

This skill creates hyperparameter tuning scripts for LightGBM models - efficient search strategies, proper cross-validation, multiple objective functions, and production-ready tuning pipelines. Parameters are prioritized in tuning order: primary (num_leaves, learning_rate, feature_fraction, bagging_fraction), secondary (min_data_in_leaf, lambda_l1, lambda_l2, min_gain_to_split), and advanced (max_depth, bagging_freq, max_bin, cat_smooth) - always tuned in the order tree structure, then regularization, then sampling. The search strategy hierarchy runs coarse grid search for major parameters, Bayesian optimization for fine-tuning, random search for exploration, and successive halving for efficiency.

Use it when a LightGBM model needs a structured, staged Optuna tuning pipeline with proper cross-validation, not a single flat grid search over all parameters at once.

class LightGBMTuner:
    def __init__(self, X, y, task_type='binary', cv_folds=5, n_trials=100):
        self.X = X
        self.y = y
        self.task_type = task_type
        self.cv_folds = cv_folds
        self.n_trials = n_trials
        self.config = self._get_task_config()

The core objective() method defines Optuna trial.suggest_int/suggest_float ranges for every primary, regularization, and advanced parameter, evaluates each trial with stratified k-fold (or plain k-fold for regression) cross-validation, trains with lgb.early_stopping, and scores with roc_auc_score for binary tasks or RMSE for regression. A multi-stage pipeline runs three progressively narrower Optuna studies - core tree-structure parameters first, then regularization using the prior stage's best params, then final fine-tuning - merging all three result dicts. Specialized variants extend the base tuner: categorical-feature tuning adds cat_smooth, cat_l2, and max_cat_threshold search ranges; time-series-aware tuning swaps in TimeSeriesSplit to preserve temporal ordering across folds; and monitored tuning reports intermediate per-fold scores to Optuna's MedianPruner so unpromising trials are pruned early. A create_production_config() step converts tuned parameters into deployment settings - force_col_wise/force_row_wise memory and speed switches based on dataset size, a capped max_bin, and fixed seeds (deterministic: True, seed, bagging_seed, feature_fraction_seed) for reproducible results.

The pipeline is built on optuna for the search itself and scikit-learn for StratifiedKFold/KFold/TimeSeriesSplit cross-validation splitting.

ML engineers tuning LightGBM models who need a staged, cross-validated Optuna pipeline with a clean production-config conversion step, not a one-shot grid search glued together by hand. Usage is a three-line pattern - initialize LightGBMTuner, call tune_with_monitoring() for pruned optimization, then create_production_config() before final training - and the closing guidance is to always validate final models on a held-out test set, monitor for overfitting throughout tuning, use feature-importance analysis to guide which parameters matter, and consider ensemble methods for critical applications.