Module 11 — Evaluation: Proving Your Model Is Actually Better

Goal: Learn to measure whether your fine-tune actually worked — rigorously, not by vibes. This is the skill that separates professionals from hobbyists. Anyone can run a training job; experts can prove the result is better and know exactly where it still fails. Without evaluation you are flying blind, and you will ship regressions you never noticed.

1. Why “it looks good” is a trap

After fine-tuning, the temptation is overwhelming: you type three prompts, the outputs look nice, and you declare victory. This is the most common and most expensive mistake in the field. Three reasons it fails:

You see what you hope to see. You picked easy prompts and read the outputs charitably. Confirmation bias is brutal here.
Improvements and regressions hide. A model can get better at your task while getting worse at something else (catastrophic forgetting — Module 05 §7), and a handful of cherry-picked prompts won’t reveal it.
You can’t improve what you don’t measure. Without a number, you can’t tell if your next change helped or hurt. Evaluation is the feedback signal for the entire iteration loop (Module 00 §2).

The professional stance: a fine-tune isn’t “done” when training finishes; it’s done when you’ve measured that it’s better on a held-out test set and checked it didn’t regress elsewhere.

2. The foundation: a held-out test set

Everything rests on the test set you reserved in Module 04 §7 — examples the model never saw during training, ideally real data (Module 09 §6). Rules that make evaluation trustworthy:

No leakage. Not one test example (or near-duplicate) may appear in training (Module 10 §4). A leak makes your model look brilliant and lies to you.
Representative. The test set must reflect the real distribution of inputs the model will face — including the hard and rare cases, not just the easy middle.
Touched rarely. Ideally you look at the final test score once, at the end. If you tune your model repeatedly against the same test set, you slowly overfit to it and your number stops being honest. (That’s why validation and test are separate — Module 04 §7.)
Big enough to be meaningful. A 10-example test set tells you almost nothing; differences are noise. Aim for at least a few hundred where feasible.

3. Always compare against baselines

A score in isolation is meaningless — “82% accuracy” is good or bad only relative to something. Always evaluate baselines on the same test set:

The base model, un-fine-tuned (with a good prompt). Did fine-tuning actually beat just prompting? Sometimes it doesn’t — and that’s vital to know before you ship a more complex system.
The base model with few-shot prompting / RAG (Module 02). Maybe a cheaper approach is good enough.
Your previous fine-tune (when iterating). Did this run improve on the last?
A strong reference model, if relevant, as an upper bound.

Rule: never report a fine-tuned model’s score without the base model’s score on the same test set right next to it. That comparison is the entire point.

4. Choosing metrics: it depends on the task

There’s no single “model score.” You pick metrics that match what you actually care about. The big split is objective vs subjective tasks.

4a. Objective tasks (there’s a right answer)

For classification, extraction, and similar, use standard, automatic metrics:

Accuracy — fraction correct. Fine when classes are balanced.
Precision / Recall / F1 — essential when classes are imbalanced or when false positives vs false negatives matter differently. (Precision: of what I flagged, how much was right? Recall: of what I should have flagged, how much did I catch? F1: their balance.)
Exact match / structural validity — for formatted output (e.g., “is it valid JSON matching the schema?”, “does the SQL run and return the right rows?”). For verifiable tasks, execution-based evaluation is the gold standard.

These are cheap, fast, and reproducible — automate them and run after every training run.

4b. Generation tasks (quality is a judgment)

For summaries, chat replies, writing, and reasoning, there’s no single right string, so it’s harder:

Reference-overlap metrics (BLEU, ROUGE, METEOR) measure word overlap with a reference answer. Use with heavy skepticism — a great answer worded differently from the reference scores low; a bad answer sharing words scores high. They’re weak proxies for quality. Useful as a rough, cheap signal, not a verdict.
Embedding similarity (e.g., BERTScore) compares meaning rather than exact words — somewhat better, still imperfect.
Perplexity (Module 03 §2) on held-out data measures how “unsurprised” the model is, but lower perplexity doesn’t guarantee better task performance. A secondary signal at best.
The real answers for generation quality are human evaluation (§5) and LLM-as-judge (§6).

The expert move: define what “good” means for your task as concrete criteria (accurate? on-brand tone? right length? cites sources? safe?) and measure those, rather than reaching for a generic metric that doesn’t capture what you care about.

5. Human evaluation: still the gold standard

For subjective quality, structured human judgment remains the most trustworthy signal. Make it rigorous, not casual:

Side-by-side / A/B comparison. Show a rater the same prompt answered by two models (e.g., base vs fine-tuned) blinded (they don’t know which is which) and ask which is better. Humans are far more reliable at comparing two outputs than scoring one in isolation. The aggregate win rate (“fine-tuned won 71% of the time”) is a strong, intuitive metric.
Rubric scoring. Define explicit criteria (1–5 on accuracy, tone, completeness, safety) so different raters are consistent and you know which dimension improved or regressed.
Multiple raters + agreement. More than one rater per item; check they agree. Low agreement means your criteria are fuzzy — fix the rubric.
Enough samples. A handful isn’t enough to trust; evaluate a few dozen to a few hundred for a meaningful read.

Human eval is slower and costlier, which is exactly why the next tool exists.

6. LLM-as-a-judge: scalable, powerful, and to be used carefully

A now-standard technique: use a strong model to evaluate your model’s outputs — scoring them against a rubric, or picking the better of two (mirroring §5’s A/B). It’s fast, cheap, and correlates reasonably with human judgment when done well. It’s also how you can re-evaluate after every change without a human in the loop.

How to do it well:

Give the judge a clear rubric and the criteria you defined (§4b). Vague prompts → noisy judgments.
Prefer pairwise comparison (“A or B, and why?”) over absolute scoring — more reliable, just like with humans.
Provide the reference/ground truth when one exists, so the judge checks correctness rather than guessing.
Ask for reasoning before the verdict, which improves judgment quality and lets you audit it.

Its known biases (an expert must account for these):

Position bias — judges often favor the first (or second) option. Mitigate by swapping order and averaging both ways.
Length/verbosity bias — judges tend to prefer longer, more elaborate answers even when worse. Watch for it; control for length.
Self-preference / style bias — a judge may favor outputs that look like its own style.
It is not ground truth. Validate the judge against human labels on a sample before trusting it at scale.

Used with these guardrails, LLM-as-judge lets you evaluate thousands of outputs quickly — the practical backbone of modern iteration. Used naively, it confidently misleads you.

7. Don’t forget: regression and safety testing

Two evaluations beginners skip and pros never do:

Regression / general-capability testing. Because fine-tuning can cause forgetting (Module 05 §7), evaluate your model on general tasks it should still handle, not just your target task. Keep a small “general capability” test set (varied reasoning, instruction-following, other formats). If those scores dropped, you traded too much (Module 05 §7) — retrain more gently or mix in general data.
Safety / robustness testing. Check the model doesn’t produce harmful, biased, or policy-violating outputs, and that it handles adversarial or malformed inputs gracefully. Especially critical for anything user-facing. Fine-tuning can weaken a base model’s safety alignment, so verify it.

A good evaluation suite therefore has three parts: (a) does it do the target task well? (b) did it keep its general abilities? (c) is it safe and robust? Report all three.

8. Building an evaluation harness (do this once, reuse forever)

Turn evaluation into a repeatable script you run after every training run. Conceptually:

def evaluate(model, test_set):
    results = {"task": [], "general": [], "safety": []}
    # 1) TARGET TASK — automatic metrics where possible (§4a) or judge (§6)
    for ex in test_set["task"]:
        out = generate(model, ex["input"])
        results["task"].append(score_task(out, ex["expected"]))  # accuracy/F1/exec/judge
    # 2) GENERAL CAPABILITY — regression check (§7)
    for ex in test_set["general"]:
        out = generate(model, ex["input"])
        results["general"].append(score_general(out, ex["expected"]))
    # 3) SAFETY — adversarial / policy prompts (§7)
    for ex in test_set["safety"]:
        out = generate(model, ex["input"])
        results["safety"].append(score_safety(out))
    return {k: mean(v) for k, v in results.items()}

# Run for EVERY model so you can compare (§3)
print("base    :", evaluate(base_model,      test_set))
print("finetune:", evaluate(finetuned_model, test_set))

The point isn’t the exact code — it’s the discipline: same test set, multiple models, three categories, one number per category, every time. Once this exists, every future change becomes a clean experiment: change one thing (Module 00 §6), re-run, compare. That is how experts make steady, provable progress instead of wandering.

You’ll also encounter standardized benchmark suites and tools (e.g., open evaluation harnesses and leaderboards) for general capabilities. They’re useful for broad sanity checks, but your own task-specific test set matters more than any public benchmark — the benchmark measures general goodness; you care about your problem.

9. Reading results honestly

Final discipline — interpret results without fooling yourself:

Mind statistical noise. Small test sets and the model’s own randomness (sampling) mean small score differences may be meaningless. When comparing close results, test more examples and consider running generation a few times.
Look at failures, not just averages. The average hides the story. Read the examples your model got wrong — they tell you exactly what data to add or fix (back to Module 10). This is the single most productive thing you can do after an eval.
Beware metric gaming. If you optimize hard against one metric, the model may learn to game it rather than truly improve (e.g., padding answers to please a length-biased judge). Use multiple metrics and human spot-checks as a cross-check.
Tie results back to the decision. The question is never “what’s the score?” but “is this good enough to ship, and where does it still fail?” Evaluation serves a decision; keep that in view.

Module 11 checklist

I can explain why “it looks good” is dangerous and why a held-out test set is essential.
I always compare against the base model (and other baselines) on the same test set.
I can pick appropriate metrics for objective vs generation tasks.
I can run rigorous human eval (blinded A/B, rubrics, agreement).
I can use LLM-as-judge well and name its biases and mitigations.
I test for regressions/forgetting and for safety, not just the target task.
I have a reusable eval harness and read failures, not just averages.

➡️ Next: Module 12 — Preference Tuning: RLHF, DPO & friends