Engineer Machine Learning Feature Pipelines

This skill covers designing, implementing, and optimizing feature engineering pipelines for ML projects, spanning feature-transformation techniques, pipeline architecture, automated feature selection, and production-ready implementations using scikit-learn, pandas, and feature-tools. Core architecture principles: modular design (separate transformers by data type - numerical, categorical, text, datetime; composition over inheritance; reversible transformations where possible; design for both batch and streaming) and data-leakage prevention (fit transformers only on training data; strict pipeline.fit()/pipeline.transform() paradigm; nested pipelines for proper cross-validation; separate target-encoding and feature-selection steps).

It provides a custom-transformer template extending scikit-learn's estimator interface:

from sklearn.base import BaseEstimator, TransformerMixin
import pandas as pd
import numpy as np

class AdvancedFeatureTransformer(BaseEstimator, TransformerMixin):
    def __init__(self, feature_columns=None, strategy='auto'):
        self.feature_columns = feature_columns
        self.strategy = strategy
        self.fitted_params_ = {}
    
    def fit(self, X, y=None):
        # Store training statistics without transforming
        if isinstance(X, pd.DataFrame):
            self.feature_names_ = X.columns.tolist()
            # Example: store quantiles for outlier detection
            for col in self.feature_columns or X.select_dtypes(include=[np.number]).columns:
                self.fitted_params_[col] = {
                    'q01': X[col].quantile(0.01),
                    'q99': X[col].quantile(0.99),
                    'median': X[col].median()
                }
        return self
    
    def transform(self, X):
        # Apply transformations using stored parameters
        X_transformed = X.copy()
        for col, params in self.fitted_params_.items():
            # Example: cap outliers
            X_transformed[col] = X_transformed[col].clip(
                lower=params['q01'], 
                upper=params['q99']
            )
        return X_transformed
    
    def get_feature_names_out(self, input_features=None):
        return self.feature_names_

A production-ready pipeline structure (FeatureEngineeringPipeline) auto-detects numeric/categorical/datetime columns, builds a numeric branch with KNNImputer plus outlier handling and RobustScaler, a categorical branch with SimpleImputer plus OneHotEncoder, combines them via ColumnTransformer, then applies SelectKBest(f_regression) feature selection and a final StandardScaler.

Feature-generation strategies cover polynomial/interaction features via PolynomialFeatures, and domain-specific engineered features - time-based (hour, day-of-week, is_weekend, quarter extracted from a timestamp column) and ratio/product combinations across numeric column pairs. Target encoding is demonstrated via a SafeTargetEncoder that computes smoothed category means (blending each category's mean with the global mean, weighted by sample count and a smoothing factor) to avoid leakage. Pipeline monitoring is covered via a FeaturePipelineAnalyzer that trains a model on transformed features, extracts feature importances (or coefficients), computes per-feature mean/std/missing-rate statistics, and generates a report of top features, low-importance features, correlated pairs, and data-quality issues.

Configuration and deployment guidance covers a structured pipeline-config dict (imputation strategy, scaling method, outlier-detection method, and interaction-creation flag for numeric features; imputation, encoding, and cardinality cap for categorical features; feature-selection method, k-best, and correlation threshold; plus CV/test-split validation settings) and a save_pipeline_artifacts function that serializes the fitted pipeline via joblib, and writes feature names and config as timestamped, versioned JSON files alongside pipeline metadata.

Use it when building or hardening a feature engineering pipeline that must avoid data leakage and generalize to production - designing custom transformers, automating feature generation, safely target-encoding categoricals, or serializing a pipeline for deployment. It is not a model-selection or hyperparameter-tuning guide - it covers the feature layer that feeds a model, not the model itself.

Given a raw DataFrame (and optionally a target for supervised feature selection or encoding), it returns a fitted scikit-learn Pipeline/ColumnTransformer object, the transformed feature matrix, and - via the analyzer and config utilities - a feature-importance report and versioned, serialized pipeline artifacts ready for production deployment.

Built entirely on scikit-learn (Pipeline, ColumnTransformer, BaseEstimator/TransformerMixin, KNNImputer, SimpleImputer, OneHotEncoder, RobustScaler, StandardScaler, SelectKBest, PolynomialFeatures, KFold) plus pandas/numpy for data handling and joblib for serialization.

ML engineers and data scientists building feature engineering pipelines that need to prevent data leakage and deploy cleanly to production.