learn — vonhas

Supervised learning fits a function from labelled examples; regression predicts continuous targets, classification predicts discrete ones. Every model is a 3-step loop: pick a hypothesis (linear, polynomial…), pick a cost (SSE/MSE/R²), then minimise. Coefficients you can interpret cost you variance you can't always afford; predictive power often comes from giving up that interpretability. Start simple, baseline against the mean, scale features when distances or coefficients are involved.

Held-out evaluation is non-negotiable: train/test splits stop you from grading the model on the answers it saw during training. Polynomial features let a linear model bend — but bend too far and you overfit. The complexity-vs-error curve is the canonical picture: train error keeps falling, test error U-shapes, the sweet spot is the bottom of that U.

A single train/test split is noisy — the score depends on which rows landed where. k-fold cross-validation averages over k splits to get a stabler estimate, and pairs naturally with hyperparameter search via `GridSearchCV`. Pipelines (`StandardScaler → estimator`) keep scaling inside the fold so the validation rows stay genuinely held-out.

Error decomposes into bias (the model is systematically wrong) and variance (the model wobbles across resamples). Regularization shrinks coefficients to trade a bit of bias for a lot less variance. Ridge (L2) pulls every coefficient toward zero; LASSO (L1) zeroes some — built-in feature selection; Elastic Net mixes the two with an α knob.

Why does regularization work? Three lenses: analytical (smaller weights ⇒ smoother predictions), geometric (the L1 diamond meets the loss contour at axes — zeros pop out; the L2 ball is round so it never quite does), and probabilistic (regularization is a prior on coefficients — Gaussian for L2, Laplacian for L1). Pick λ from the cross-validated dip.