Module 09 — Evaluating Prompts (Evals)

Goal: Learn to measure whether a prompt is good — systematically, not by vibes. This is the single biggest skill separating hobbyists from professionals. If you can’t measure quality, you can’t improve it or know when a change made things worse.

9.1 Why evaluation is the expert skill

Beginners tweak a prompt, eyeball one or two outputs, and declare victory. Then in production it fails on inputs they never tried. Professionals build evals (evaluations): a repeatable way to score a prompt across many representative inputs.

Why it matters:

LLMs are non-deterministic. The same prompt can give different outputs. One good run proves nothing.
Changes have side effects. Fixing one case often breaks another. Without a test set, you can’t tell.
“Better” must be defined. Better how — accuracy? tone? format? You need explicit criteria.
Models and prompts change over time (new model versions, edits). Evals catch regressions.

Core principle: Treat prompts like code. Code has tests; prompts need evals. “It looked fine when I tried it” is not engineering.

9.2 The anatomy of an eval

An eval has three parts:

A dataset of test cases: representative inputs, ideally with expected outputs or success criteria.
A scoring method: how you decide if an output is good (exact match, rules, a rubric, an LLM judge, or a human).
A metric/report: aggregate scores (e.g., “87% correct,” “average rubric score 4.2/5”) so you can compare versions.

[Test cases] ──run prompt──► [Outputs] ──score──► [Metrics] ──compare versions──► improve

9.3 Building a test set

Your eval is only as good as its cases. Guidelines:

Use real, representative inputs. Pull from actual usage/logs if you can, or write realistic ones.
Cover the distribution: common cases, rare cases, and edge cases (empty input, very long input, ambiguous input, adversarial input, multiple languages, missing fields).
Include known-hard cases — the ones your prompt keeps getting wrong. These are the most valuable tests.
Start small, grow over time. Even 20–50 well-chosen cases beat zero. Add a new test every time you find a failure (“regression test”).
Where possible, label the expected answer or define a clear pass condition. For subjective tasks, define a rubric (9.5).
Keep a held-out set you don’t tune against, to check you’re not overfitting your prompt to your test cases.

Store cases simply — a spreadsheet or JSON file of {input, expected/criteria} works great.

9.4 Scoring methods (from cheapest to richest)

1. Exact match / programmatic checks

For tasks with a definite right answer or format, check in code:

Classification: does the label equal the expected label?
Extraction/JSON: is it valid JSON? Are the right fields present with right values? (Module 05)
Does the output contain/avoid certain strings? Is it within a length limit? Does it match a regex/format?

Cheap, fast, objective, fully automatable. Use whenever the task allows. Common metrics: accuracy, precision/recall/F1 (for classification — precision = of the items you labeled X, how many were truly X; recall = of all true X, how many you caught; F1 balances both).

2. Reference-based similarity

Compare output to a “golden” reference answer. Classic text metrics (BLEU, ROUGE) measure word overlap — useful for translation/summarization but crude (they miss meaning). Embedding similarity (semantic closeness, like in RAG) is better for “is this roughly the same meaning as the reference?“

3. LLM-as-judge (very popular)

Use a (often stronger) model to grade outputs against a rubric. Great for subjective qualities (helpfulness, tone, coherence) that code can’t measure and human grading is too slow for. Example judge prompt:

You are grading an answer. Score 1–5 on each criterion and give one-line reasons.
Criteria: (a) factual accuracy vs. the reference, (b) completeness, (c) clarity,
(d) follows the requested format.

Question: {q}
Reference answer: {ref}
Answer to grade: {output}

Return JSON: {"accuracy":n,"completeness":n,"clarity":n,"format":n,"notes":"..."}

LLM-judge cautions (know these — interviewers ask):

Judges are biased: they can favor longer answers, the first option in a pairwise test (position bias), or their own style. Mitigate by randomizing order, using clear rubrics, and pairwise comparison (“is A or B better?”) which is often more reliable than absolute scores.
Validate the judge against human labels on a sample — make sure the judge agrees with humans before trusting it at scale.
Use a strong model as judge, ideally different from the one being tested when possible.

4. Human evaluation

The gold standard for nuanced quality and the thing you ultimately optimize for. Expensive and slow, so use it to: validate your automated metrics, spot-check, and judge high-stakes or subtle cases. Make it rigorous with clear rubrics and multiple raters. Pairwise “which is better, A or B?” is easier and more consistent for humans than absolute scores.

Typical real setup: programmatic checks for everything measurable + LLM-as-judge for subjective quality at scale + periodic human review to keep the automated metrics honest.

9.5 Rubrics: defining “good” concretely

For open-ended tasks, write a rubric — explicit criteria with descriptions of what each score means. Example for a support-reply task:

Accuracy (0-2): 0 = wrong/misleading, 1 = partially correct, 2 = fully correct.
Helpfulness (0-2): resolves the user's need / gives clear next step.
Tone (0-1): warm and professional.
Safety (0-1): no policy violations, no fabricated info.
Format (0-1): follows required structure/length.

A rubric forces you to articulate what you actually want (often clarifying the prompt itself), makes scoring consistent across people and across LLM-judges, and turns “feels good” into numbers you can track.

9.6 The improvement loop with evals

1. Define success criteria + build a test set with expected outputs/rubric.
2. Run the current prompt over all cases; score; record the baseline metric.
3. Make ONE change to the prompt (a hypothesis: "adding an example will fix X").
4. Re-run the full eval. Did the metric improve? Did anything regress?
5. Keep the change if it helps overall; revert if it doesn't. Log the result.
6. Add any new failures you discover as permanent test cases.
7. Repeat. Track metrics over versions.

This is the scientific method applied to prompting: hypothesis → experiment → measure → keep/discard. Change one thing at a time so you know what caused the difference. Without the full re-run, you’d never notice that fixing case 7 broke case 3.

9.7 What to measure besides quality

Production evals track more than correctness:

Cost: tokens/dollars per request (Module 11).
Latency: how long responses take.
Reliability: valid-format rate, tool-call success rate, error rate.
Safety: rate of policy violations, refusals on benign requests (over-refusal), jailbreak resistance (Module 10).
Consistency: variance of output across repeated runs (run each case a few times).

A faster, cheaper prompt that’s 1% less accurate may be the better choice — you can only make that call if you measure all dimensions.

9.8 Tooling (awareness)

You can run evals with a simple script (loop over a CSV, call the model, score, print metrics) — and you should be able to do this by hand. Beyond that, eval frameworks and platforms exist (e.g., OpenAI Evals, promptfoo, LangSmith, Braintrust, Ragas for RAG, and others) that manage datasets, run comparisons across prompt/model versions, and visualize results. Tools change fast; the concepts in this module are what last. Learn the concepts first; pick a tool when you need scale.

9.9 Special case: evaluating RAG and agents

RAG: evaluate retrieval and generation separately. Retrieval: did the right chunks come back (precision/recall of retrieved docs)? Generation: is the answer grounded in (faithful to) the retrieved context, and does it actually answer the question? Many RAG failures are retrieval failures (Module 08) — separating the two tells you where to fix.
Agents: evaluate not just the final answer but the trajectory — did it pick the right tools, in a sensible order, without wasteful loops, within budget? Log every step and score the process, not only the outcome.

9.10 Common pitfalls

Pitfall	Fix
Judging by one or two outputs	Build a real test set; run all cases
Testing only easy/happy-path inputs	Add edge cases and known-hard cases
Changing several things at once	Change one variable per experiment
Overfitting the prompt to your tests	Keep a held-out set; use realistic data
Blindly trusting an LLM judge	Validate it against human labels; watch for biases
No regression tests	Add every new failure as a permanent case
Ignoring cost/latency/safety	Measure them alongside quality
”Vibes-based” iteration	Define metrics; track them across versions

Exercises

Make a test set. For a sentiment classifier (positive/negative/neutral), write 15 test cases with expected labels, deliberately including 3 tricky ones (sarcasm, mixed sentiment, neutral-but-negative-words).
Score programmatically. Run your classifier prompt over the 15 cases and compute accuracy by hand. Which cases failed? Why?
Write a rubric. Create a 5-criterion rubric (with score meanings) for evaluating a “summarize this email” prompt.
LLM-as-judge. Write a judge prompt that scores two candidate summaries against your rubric and picks the better one. Note one bias you’d worry about and how you’d reduce it.
Improvement loop. Take a failing case from #2, make ONE change to the prompt to fix it, re-run ALL 15 cases, and check whether overall accuracy went up and whether anything regressed.

Key takeaways

Evaluation is what makes prompt engineering an engineering discipline. Measure, don’t vibe.
An eval = test set + scoring method + metrics. Build a representative test set including edge and known-hard cases.
Scoring ranges from programmatic checks (cheap, objective) to LLM-as-judge (scalable, subjective, but biased — validate it) to human review (gold standard).
Improve via a one-change-at-a-time loop, re-running the full eval to catch regressions; add every new failure as a permanent test.
Measure cost, latency, reliability, and safety too. Evaluate RAG retrieval/generation separately and agents by their trajectory.

Next: Module 10 — Safety, Security & Prompt Injection.