Module 03 — Training Fundamentals: How Learning Actually Happens

Goal: Open the black box of “training” so that every knob you’ll later turn (learning rate, epochs, batch size) means something. We’ll explain loss, gradients, and gradient descent with zero scary math — just clear pictures. If you understand this module, you’ll debug training problems that stump people who skipped it.

1. Learning = reducing a measure of wrongness

All of model training, including fine-tuning, is one idea repeated millions of times:

The model makes a prediction.
We measure how wrong it was with a single number.
We adjust the weights a tiny bit to make that number smaller.
Repeat.

That “how wrong” number is the loss. Training is just minimizing the loss. Everything else is detail. Let’s make each step concrete.

2. Loss: putting a number on “how wrong”

Recall from Module 01 that the model outputs a probability for every possible next token. Suppose the correct next token is “Paris,” and the model said:

Case A: Paris → 0.91 (confident and right). Good. Low loss.
Case B: Paris → 0.10 (right answer got low probability). Bad. High loss.

The standard loss for language models is cross-entropy loss. You don’t need the formula, just its behavior:

Cross-entropy loss is high when the model gave low probability to the correct token, and low when it gave high probability. It punishes confident wrongness severely.

The loss for a whole training example is the average of this over every token the model had to predict. The loss for a training run is tracked as a curve over time — and watching the loss curve go down is how you know learning is happening. A flat or rising loss means something is wrong.

Perplexity (a friendly cousin)

You’ll see perplexity reported too. It’s just the loss transformed into a more intuitive scale: roughly, “on average, how many tokens was the model wavering between?” Perplexity of 1 = perfect certainty and correctness; higher = more confused. Lower is better, like loss. Mentioned here so it’s not mysterious later.

3. Gradient descent: how the weights actually change

We have a number (loss) we want to make small, by adjusting billions of weights. How? With gradient descent — the engine of essentially all deep learning.

The hill analogy (the one to remember)

Imagine the loss as a landscape of hills and valleys. Your model’s current weights place you at some spot on this landscape. High ground = high loss (bad). Valleys = low loss (good). You want to walk downhill to a valley.

You’re blindfolded, so you can’t see the whole landscape. But you can feel the slope under your feet — which direction goes downhill, and how steeply. That local slope is the gradient.

Gradient descent is simply:

Feel the slope (compute the gradient — the direction of steepest increase in loss).
Take a small step in the opposite direction (downhill).
Repeat until you’re in a valley (low loss).

That’s the whole algorithm. “Training a neural network” is millions of tiny downhill steps.

Where does the gradient come from? (Backpropagation)

The gradient tells us, for each individual weight, “if you nudge this weight up a hair, does the loss go up or down, and by how much?” Computing this efficiently for billions of weights at once is done by an algorithm called backpropagation (“backprop”). You will essentially never implement it — PyTorch does it automatically — but know the name and what it produces: a gradient for every weight, saying which way to nudge it.

Why this matters for fine-tuning memory: to compute and store a gradient for every weight, plus extra bookkeeping per weight (the optimizer state, see §6), you need several times the model’s size in memory. This is the core reason full fine-tuning is so expensive — and exactly the cost LoRA sidesteps by only computing gradients for a tiny number of weights. You’ll appreciate LoRA much more having understood this.

4. Learning rate: the size of each step (the most important knob)

If gradient descent is “take a step downhill,” the learning rate is how big each step is. It is the single most important hyperparameter you’ll set, so build real intuition here.

Too large → you take giant leaps and overshoot the valley, bouncing around or flying off entirely. Symptom: loss spikes, becomes erratic, or turns into NaN (“not a number” — the model exploded). The model can be destroyed in a few steps.
Too small → you inch downhill so slowly that training takes forever, or you get stuck on a tiny bump and never reach a good valley. Symptom: loss barely moves.
Just right → loss descends smoothly and steadily.

loss
 │＼                      too small: barely moves
 │ ＼___________________
 │
 │＼  ／＼  ／＼  ／        too large: bounces / spikes
 │ ＼／  ＼／  ＼／
 │
 │＼                       just right: smooth steady descent
 │ ＼＿
 │   ＼＿＿＿＿＿＿＿＿＿
 └─────────────────────► training steps

Typical learning rates for fine-tuning are small numbers like 2e-5 (0.00002) for full fine-tuning, and somewhat larger like 1e-4 to 2e-4 for LoRA (because LoRA trains fewer, fresh parameters and can tolerate bigger steps). Don’t memorize these as law — but know the ballpark, and know that when in doubt, lower the learning rate: it’s the most common fix for unstable training.

Learning-rate schedules and warmup

In practice we don’t keep the step size constant. Two refinements you’ll see and should recognize:

Warmup: start with a tiny learning rate and ramp up over the first few hundred steps. This prevents a violent, destabilizing first step into a model that was carefully pretrained. Almost always used.
Decay (schedule): gradually shrink the learning rate over the run (e.g., cosine or linear decay), so you take big confident steps early and small careful steps near the end as you settle into a valley.

You’ll just set warmup_steps and lr_scheduler_type="cosine" in a config; now you know what they do and why.

5. Epochs, batches, and steps: the vocabulary of “how long”

These three words describe how training data flows through training. Get them straight once and you’re set.

Batch: We don’t show the model one example at a time (too noisy and slow). We show a small group — a batch — compute the average loss over it, and take one downhill step. Batch size = how many examples per step. Bigger batches give smoother, more reliable steps but use more memory.
Step (iteration): One downhill update of the weights, using one batch. “Training for 1,000 steps” = 1,000 weight updates.
Epoch: One full pass through your entire training dataset. If you have 2,000 examples and batch size 8, one epoch = 250 steps.

How many epochs for fine-tuning?

For fine-tuning, fewer epochs than beginners expect — often just 1 to 3. Why so few? Because you’re starting from an already-smart model and only nudging it. Train too long on a small dataset and you hit the central danger of the next section.

Gradient accumulation (a memory trick you’ll use constantly)

What if you want the stability of a big batch (say 32) but your GPU can only fit 4 examples at a time? Gradient accumulation lets you process 4 examples, remember their gradients, do that 8 times, and only then take one step — giving you the effect of batch size 32 (4 × 8) with the memory of 4. You’ll set per_device_train_batch_size=4 and gradient_accumulation_steps=8. The effective batch size is the product. This trick is everywhere in low-resource fine-tuning; remember the multiplication.

6. The optimizer: a smarter way to step

Plain gradient descent (“step directly downhill by the learning rate”) works, but we can do better with an optimizer — an algorithm that decides how to apply the gradients more cleverly. The near-universal default is Adam (and its memory-friendlier variants AdamW, 8-bit Adam).

The one intuition to keep: Adam gives each weight its own adaptive step size and adds momentum (like a ball rolling downhill that keeps some speed through small bumps), making training faster and more stable than naive gradient descent.

The catch — and it matters for fine-tuning budgets — is that Adam stores two extra numbers per weight (its bookkeeping, the “optimizer state”). So for full fine-tuning you’re holding the weights, their gradients, and two optimizer values each — roughly 4× the model’s size in memory just for the math, before counting the data flowing through. This is, again, exactly the cost LoRA avoids by only optimizing a handful of new parameters. (Module 08’s QLoRA squeezes it further.)

7. Overfitting: the central danger of fine-tuning

This is the failure mode you’ll fight most, so understand it deeply.

Overfitting is when the model memorizes your training examples instead of learning the general pattern. It looks great on the data it trained on and fails on anything new — like a student who memorized last year’s exact exam answers and is lost when the questions change.

Why it’s especially a fine-tuning risk: fine-tuning datasets are often small (hundreds to thousands of examples), and a model with billions of parameters can memorize a small dataset frighteningly easily, especially over too many epochs.

How you detect it: the train/validation split

Before training, you split your data: most for training, a held-out slice (say 5–10%) for validation that the model never trains on. During training you watch both losses:

loss
 │
 │ ＼                              ← training loss keeps dropping
 │  ＼＿
 │    ＼＿＿＿＿＿＿＿＿＿＿
 │
 │ ＼                              ← validation loss drops, then RISES
 │  ＼＿＿＿＿＿＿
 │           ＿＿／￣￣      ⟵ OVERFITTING STARTS HERE
 │          ／
 └──────────┼────────────────► epochs
            ▲
     stop training around here ("early stopping")

Training loss falling + validation loss falling → genuine learning.
Training loss falling + validation loss rising → overfitting. The model is memorizing. Stop.

The trick of halting when validation loss bottoms out is called early stopping, and it’s one of your main defenses. Others (more in Module 05): fewer epochs, more/varied data, regularization like weight decay and dropout, and lower-capacity methods like LoRA which — pleasant bonus — overfit less readily because they have far fewer trainable parameters.

The opposite problem: underfitting

If both losses stay stubbornly high, the model hasn’t learned enough — underfitting. Causes: learning rate too low, too few epochs, too little/too-weak capacity, or data too noisy to learn from. The fix is usually the opposite of the overfitting fix (train more/harder), which is why you need the validation curve to tell them apart.

8. Putting it together: the training loop in pseudocode

Everything above, in the shape you’ll actually meet it. You’ll rarely write this by hand (trainers do it for you), but seeing it once demystifies the libraries:

for epoch in range(num_epochs):
    for batch in training_data:                 # one batch at a time
        predictions = model(batch.inputs)       # forward pass: predict
        loss = cross_entropy(predictions, batch.correct_tokens)  # how wrong?
        loss.backward()                          # backprop: get gradients
        optimizer.step()                         # nudge weights downhill (Adam + LR)
        optimizer.zero_grad()                    # reset for next batch

    validation_loss = evaluate(model, validation_data)   # check for overfitting
    print(epoch, validation_loss)                        # watch this number!

Read it twice. Forward pass → loss → backward pass → step → repeat, with a validation check each epoch. When a 300-line fine-tuning script intimidates you later, remember it’s this skeleton wrapped in convenience.

Module 03 checklist

I can explain loss as “a number measuring wrongness” and that training minimizes it.
I can describe gradient descent with the blindfolded-hill analogy.
I can explain what the learning rate does and the symptoms of too-high vs too-low.
I know warmup and decay/cosine schedules at an intuition level.
I can define epoch, batch, step, and gradient accumulation (and effective batch size).
I understand why Adam/full fine-tuning is memory-hungry (and why LoRA isn’t).
I can explain overfitting, the train/validation split, and early stopping.

➡️ Next: Module 04 — Data Foundations