Overfitting

Training error decreases, but test error increases. This happens when a model is too large relative to its dataset. The scaling laws implicitly address overfitting by showing that large models trained on proportionally large datasets don't overfit.

When the model memorizes training data instead of learning general patterns; good on train, bad on new data. (M03)