Generate Advanced Polynomial Features

Provides expert guidance for polynomial feature generation in machine learning - creating polynomial expansions, interaction terms, and advanced feature transformations while managing complexity and overfitting risk.

Use this skill when adding polynomial or interaction terms to a feature set, building a scikit-learn pipeline that tunes polynomial degree via grid search, selecting the most valuable polynomial terms with regularization, or generating polynomial features efficiently for large datasets. Not a fit for datasets where feature relationships are already well captured by tree-based models that learn interactions natively, or for applying polynomial expansion directly to categorical variables.

Core principles: start with degree-2 expansion and evaluate performance before increasing complexity, focus interaction terms on domain-meaningful pairs rather than exhaustive combinations, balance feature richness against computational cost for high-dimensional data, always pair polynomial features with regularization, and apply feature selection to find the most valuable terms.

Provides a create_polynomial_features function wrapping scikit-learn's PolynomialFeatures with degree/interaction-only/include-bias options and feature name extraction, and a custom AdvancedPolynomialFeatures class that generates all polynomial terms then selects a subset based on correlation with the target above a threshold (optionally capped to a max feature count).

Domain-specific feature engineering shows a function creating squared/cubed terms for continuous variables, explicit interaction pairs (product and ratio) between named feature pairs, and categorical-numeric interaction terms via one-hot encoding multiplied by squared numeric values. A complete pipeline example chains StandardScaler, PolynomialFeatures, SelectKBest (f_regression), and Ridge regression, tuned via GridSearchCV over degree, interaction-only, k-best, and alpha ranges.

Performance guidance includes a batch_polynomial_features function that fits the transformer on the first batch and transforms subsequent batches to handle large datasets memory-efficiently. Regularization/selection guidance includes an analyze_polynomial_importance function using LassoCV to identify non-zero-coefficient polynomial features, sorted by absolute coefficient magnitude.

Best practices recommend starting with degree 2 and interaction_only=True to avoid combinatorial explosion, always standardizing features before polynomial expansion, using Ridge/Lasso/Elastic Net to handle multicollinearity, monitoring overfitting via cross-validation, using interaction_only=True or custom selection for datasets with over 20 features, and using time-series or stratified splits appropriately. Common pitfalls include creating too many features without regularization, ignoring multicollinearity, skipping feature scaling, using degrees above 4 without careful validation, applying polynomial expansion directly to categorical variables, and forgetting to transform test data with the training-fitted transformer.

Built on scikit-learn's PolynomialFeatures, StandardScaler, SelectKBest, Ridge, LassoCV, and GridSearchCV for pipeline construction and hyperparameter tuning.

ML engineers and data scientists adding polynomial or interaction features to a model who need concrete, regularization-aware implementation patterns rather than raw PolynomialFeatures usage without overfitting safeguards.

poly = PolynomialFeatures(degree=degree, interaction_only=interaction_only)
X_poly = poly.fit_transform(X)