Test information sneaking into training — duplicate rows, scaler fit on the full dataset, features built from the target. Inflates scores and breaks generalisation.
Cross-validation splitter: chop data into k slabs, train on k−1, validate on 1, rotate. Common k = 5 or 10. Set `shuffle=True` unless the row order means something.
Regularisation that penalises the sum of absolute coefficients: `λ Σ |β|`. Drives some coefficients to exactly zero — automatic feature selection. Slower to fit (coordinate descent).
Regularisation that penalises the sum of squared coefficients: `λ Σ β²`. Shrinks everything toward zero but doesn't zero anything out. Fast (closed-form solution).
The per-example error term inside the cost function. Squared error for regression, log-loss for classification. Often used interchangeably with 'cost' in casual usage.
Chains preprocessing and an estimator into a single object. Critical for cross-validation: each fold's training rows refit the preprocessor independently.
Expand each feature with its powers and cross-products. `PolynomialFeatures(degree=2)` turns `[x₁, x₂]` into `[x₁, x₂, x₁², x₁x₂, x₂²]`. The model stays linear in the coefficients.
Coefficient of determination. Fraction of variance in y explained by the model. 1 = perfect, 0 = no better than predicting the mean, negative = worse than that.
Fit, drop the lowest-importance feature, refit, repeat. `RFE` needs a fixed target count; `RFECV` picks the count by cross-validation. Works on any estimator with `.coef_` or `.feature_importances_`.
Any technique that constrains a model to reduce variance — typically by penalising coefficient magnitudes. The α / λ knob controls strength; pick it via cross-validation.
Model is too rigid to capture the underlying pattern. Both train and test error are high. Fix: more features, polynomial expansion, lower regularisation.
Sensitivity to which exact training rows you used. High-variance models overfit — tiny train error, large test error, very different fits across resamples.