Generate Robust XGBoost Training Scripts

Builds robust, production-ready XGBoost training scripts as a structured XGBoostTrainer class covering data preparation (categorical label encoding, stratified train/test split), hyperparameter configuration (task-specific defaults - max_depth=6, learning_rate=0.1, n_estimators=100, subsample/colsample_bytree=0.8, tree_method='hist' for speed on large datasets - switched between binary:logistic/logloss for classification and reg:squarederror/RMSE for regression), and hyperparameter search via GridSearchCV or RandomizedSearchCV over a defined parameter grid (depth, learning rate, estimator count, subsampling ratios, L1/L2 regularization) with 5-fold cross-validation. Training uses xgb.DMatrix with early stopping (default 10 rounds, up to 1000 boosting rounds) and tracks evaluation results across train/validation sets; a separate cross-validation method runs xgb.cv with early stopping and a fixed seed for reproducibility. Evaluation reports accuracy and a full classification report, plots feature importance for the top 20 features, and returns importance scores by weight; models save via joblib or XGBoost's native format, with feature names persisted alongside the model. A main() pipeline function chains all of this together with logging and exception handling: load data, prepare it, tune hyperparameters, train with a held-out validation split, evaluate, and save.

Use it to stand up a complete XGBoost training workflow - from raw tabular data through a tuned, evaluated, saved model - rather than writing ad hoc training code per project, especially when you need reproducible hyperparameter search, early stopping, and feature-importance reporting built in from the start.

Input is a labeled tabular dataset - a CSV with a target column, per the pipeline's main(). Output is a trained XGBoost model saved via save_model (joblib or XGBoost's native format), its feature names, evaluation metrics from evaluate_model, cross-validation results from a separate cross_validate method, and the best hyperparameters found by hyperparameter_search.

Built on xgboost (DMatrix, xgb.train, xgb.cv, plot_importance), scikit-learn (train_test_split, GridSearchCV/RandomizedSearchCV, LabelEncoder, accuracy_score, classification_report), joblib for model persistence, and Python's logging module for pipeline observability.

For ML engineers who want a reusable, production-oriented XGBoost training script rather than a notebook-only experiment. Advanced tips covered: tree_method='gpu_hist' for GPU acceleration on large datasets, max_bin=256 for memory-efficient categorical handling, scale_pos_weight for imbalanced classes, custom evaluation metrics via the feval parameter, SelectFromModel-based feature selection for high-dimensional data, and saving intermediate models during long training runs.

def get_default_params(self, task_type='classification'):
    """Get optimized default parameters based on task type"""
    base_params = {
        'max_depth': 6,
        'learning_rate': 0.1,
        'n_estimators': 100,
        'subsample': 0.8,
        'colsample_bytree': 0.8,
        'random_state': 42,
        'n_jobs': -1,
        'tree_method': 'hist',  # Faster for large datasets
        'enable_categorical': True  # XGBoost 1.5+
    }
    
    if task_type == 'classification':
        base_params.update({
            'objective': 'binary:logistic',
            'eval_metric': 'logloss'
        })
    elif task_type == 'regression':
        base_params.update({
            'objective': 'reg:squarederror',
            'eval_metric': 'rmse'
        })
    
    return base_params