Module 12 — Capstone Projects & Continued Learning

Goal: Cement mastery by building. Reading teaches concepts; projects build skill and a portfolio. This module gives you graded projects (beginner → expert), a practice routine, a self-assessment checklist, and where to keep learning.

12.1 Why projects, not more reading

You now have the full conceptual map: foundations, prompt anatomy, system prompts, core techniques, structured output, tool calling, chaining, RAG/agents, evals, safety, and production. The gap between “knows about” and “expert” is closed only by repeatedly applying these to real problems and measuring the results. The people considered experts are the ones with a track record of shipped, evaluated prompt systems — and the stories of what broke and how they fixed it.

Rule: For every project below, write an eval set (Module 9) and iterate against it. A project without evaluation is practice without feedback.

12.2 Capstone projects (progressive)

Pick projects slightly above your comfort level. Each lists the skills it exercises.

Beginner

The Universal Summarizer. A prompt that summarizes any pasted text into 3 bullets + a one-line TL;DR, ignoring instructions embedded in the text. Skills: anatomy, delimiters, basic safety, format control. Stretch: add tone options (formal/casual) and a length control.
Smart Classifier. Classify customer messages into categories with a confidence score, output as JSON. Build a 20-case eval set and measure accuracy. Skills: few-shot, structured output, evaluation.
Persona Tutor. A system prompt for a Socratic tutor in a subject you like — asks questions rather than giving answers, adapts to the learner, refuses to just do homework. Skills: system prompts, persona, guardrails.

Intermediate

Structured Data Extractor. Turn messy text (resumes, invoices, or emails) into validated JSON against a schema, with a repair-on-failure loop. Skills: structured output, validation, schemas.
3-Stage Content Pipeline. Outline → Draft → Polish as a chain (run by hand or in code), with a critique step. Compare quality to a single-prompt baseline using a rubric. Skills: chaining, self-critique, evaluation.
Mini RAG Q&A. Take ~10 documents (notes, a manual, FAQs). Build a retrieval-then-answer prompt that cites sources and says “I don’t know” when unsupported. Test answerable and unanswerable questions; evaluate retrieval and answers separately. Skills: RAG, grounding, eval of RAG.

Advanced

Tool-Using Assistant. An assistant with 2–3 real tools (e.g., calculator, a fake “database” lookup, a web search) that decides when to call them, handles tool errors, and never fabricates. Skills: tool calling, ReAct loop, error handling.
Evaluated Prompt System. Pick any task; build a 50-case eval set (incl. edge + adversarial cases), an LLM-as-judge scorer validated against your own human labels, and a versioned improvement log showing measured gains across 3 prompt versions. Skills: evaluation depth, LLM-as-judge, versioning.
Hardened Bot. Take project 7 and red-team it: jailbreaks, direct + indirect injection, leak attempts. Add layered defenses and prove (with tests) the attacks now fail. Skills: security, prompt injection defense, red-teaming.

Expert / portfolio-grade

End-to-End Mini Product. Combine it all: a system prompt + RAG over your data + tools + a small chain/agent loop + structured output + an eval suite + safety guardrails + a cost/latency report. Write it up: the problem, your design choices, eval results, failures you hit, and how you fixed them. This single project demonstrates everything and is genuinely impressive in a portfolio or interview.

For projects 4–10, do them in code (Python is the common choice) using your preferred provider’s SDK. You don’t need to be a strong programmer — the orchestration logic is mostly “call model, check result, call again.”

12.3 A weekly practice routine

To keep improving after this course:

Prompt journal (ongoing): every time a prompt works well or fails, save it with a note on why. Review monthly. This becomes your personal playbook and is where real intuition forms.
Reverse-engineer good prompts: when a product gives a great AI response, hypothesize the prompt/system behind it. When system prompts leak or are published, study them.
One technique per week: deliberately practice a single technique (few-shot, CoT, a chaining pattern) across several tasks until it’s second nature.
Teach it: explain a concept (CoT, RAG, injection) to someone else or in a short write-up. Teaching exposes the gaps in your understanding.
Read failures: spend time on why prompts fail, not just successes. Diagnosing failure is the core expert skill (Module 9).
Stay current, skeptically: models and tools change monthly. Learn the new capabilities, but anchor on the durable principles in this course rather than chasing every “one weird trick.”

12.4 Self-assessment: are you an expert?

You’re approaching mastery when you can, without notes:

Explain why a model hallucinates and name three ways to reduce it.
Take a vague request and turn it into a precise prompt using the six building blocks.
Decide when to use zero-shot vs. few-shot vs. CoT vs. chaining vs. an agent — and justify it.
Get reliable JSON out of a model and validate it in code.
Describe the tool-calling loop and who executes the tool.
Design a RAG prompt that resists hallucination and explain why most RAG failures are retrieval failures.
Build an eval set and an improvement loop, and explain LLM-as-judge biases.
Explain prompt injection (direct vs. indirect) and design layered defenses.
Reason about cost/latency/quality trade-offs and model selection for a given use case.
Most importantly: diagnose why a given output is wrong and know exactly which lever to pull to fix it.

If you can do all of these and have shipped a few evaluated projects, you are operating at an expert level. The remaining growth is depth, breadth of domains, and reps.

12.5 Where to keep learning (durable sources)

The field moves fast, so prefer primary, regularly-updated sources over static blog posts:

Official model provider docs and prompting guides (Anthropic, OpenAI, Google, etc.) — they document each model’s quirks, tool-calling, structured-output features, and best practices. These are the most accurate and current.
Provider “cookbooks”/example repos — runnable code for tool use, RAG, evals, agents.
Open guides: community resources like the “Prompt Engineering Guide” (promptingguide.ai) and DAIR.ai materials collect techniques and papers.
Foundational papers (read for depth): Chain-of-Thought; Self-Consistency; ReAct; Tree of Thoughts; Retrieval-Augmented Generation; Toolformer; the InstructGPT/RLHF paper (how alignment works). You don’t need the math — read the ideas and examples.
Security: OWASP “Top 10 for LLM Applications,” and NIST AI risk materials.
Eval tooling docs: promptfoo, LangSmith, Braintrust, Ragas (for RAG) — for hands-on evaluation practice.
Practice surfaces: build with any provider’s free tier/playground; reproduce techniques from papers; contribute to or read open-source LLM apps.

Because details change after this course was written, when you need current specifics (today’s model names, prices, exact API parameters, newest features), check the official docs directly rather than trusting any static guide — including this one.

12.6 The throughline (what to remember above all)

Strip away the jargon and prompt engineering is one idea applied relentlessly:

The model continues text plausibly. Your job is to shape the context — instructions, examples, structure, and supplied information — so the most plausible continuation is the correct, useful, safe output. Then measure whether it actually is, and iterate.

Everything — system prompts, few-shot, CoT, JSON schemas, tools, RAG, chaining, agents, evals, safety, production ops — is a specific way of shaping context and verifying results. Master that mindset and the specific techniques become tools you reach for naturally.

You’ve now got the full map. Go build, measure, break things, fix them, and keep a journal. That’s the whole path from zero to mastery.

Final exercises

Ship one capstone. Choose project #6 (Mini RAG) or #8 (Evaluated System) and complete it end-to-end, including an eval set and a written record of three measured improvements.
Portfolio write-up. Document that project: problem, design (which techniques and why), eval results, a failure you hit, and the fix. This is your proof of expertise.
Teach-back. Write a one-page explainer of “prompt injection and how to defend against it” for a non-technical colleague. If you can teach it clearly, you own it.
Set your next goal. Pick the one area above where you’re weakest and plan three deliberate practice sessions on it this month.

Key takeaways

Mastery comes from building evaluated projects, not more reading. Always pair a project with an eval set.
Work through progressively harder capstones; the end-to-end mini product demonstrates the whole skill set.
Keep a prompt journal, practice one technique at a time, study failures, and teach others.
Prefer official, current provider docs for specifics; read foundational papers for depth.
The throughline: shape context so the right output is most likely, then measure and iterate. That mindset is the real mastery.

Congratulations — you’ve completed the curriculum. Return to the roadmap any time, and start building.