learn — vonhas

Section 1 · Regression

Foundations

Executive Summary

Supervised learning fits a function from labelled examples; regression predicts continuous targets, classification predicts discrete ones. Every model is a 3-step loop: pick a hypothesis (linear, polynomial…), pick a cost (SSE/MSE/R²), then minimise. Coefficients you can interpret cost you variance you can't always afford; predictive power often comes from giving up that interpretability. Start simple, baseline against the mean, scale features when distances or coefficients are involved.

Key Concepts

Code cheat-sheet

from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error, r2_score
 
lr = LinearRegression().fit(X_train, y_train)
y_pred = lr.predict(X_test)
 
print("MSE:", mean_squared_error(y_test, y_pred))
print("R²: ", r2_score(y_test, y_pred))

Math

SSE=∑i=1n(yi−y^i)2\text{SSE} = \sum_{i=1}^{n} (y_i - \hat{y}_i)^2

Sum of squared errors. Lower = closer fit. Sensitive to outliers because errors are squared.

MSE=1n∑i=1n(yi−y^i)2\text{MSE} = \frac{1}{n}\sum_{i=1}^{n} (y_i - \hat{y}_i)^2

SSE averaged over n — comparable across datasets of different sizes.

R2=1−∑(yi−y^i)2∑(yi−yˉ)2R^2 = 1 - \frac{\sum (y_i - \hat{y}_i)^2}{\sum (y_i - \bar{y})^2}

Fraction of variance explained. 1 = perfect; 0 = no better than the mean; negative = worse than the mean.

Self-check

Gotchas

Baseline first. Always compute the score of predicting the mean. If your model can't beat that, the features are weak or the model is mis-specified.

Interpretability ≠ accuracy. Linear coefficients are readable but often miss non-linear structure. Don't keep a linear model just because it's interpretable if the residuals are clearly curved.