Step 8 — Capstone

Released with Lecture 8 · open-ended (~4+ hours) · Starts from: your full stack (or solutions/step-07.ipynb).

Goal

You now own a complete, trained, tokenized language model built entirely by your own hand. The capstone is where you turn it toward one question of your choosing and present the result in a five-minute showcase on the last day. Pick one of the three tracks below, or propose your own.

Each track connects your code to one strand of Lecture 8 and beyond. Depth over breadth: a single clean result with a plot and a paragraph of mathematics beats a scattershot survey.

Deliverable (all tracks)

A notebook that runs top to bottom from a Step 7 checkpoint, containing: one clearly stated question, the experiment answering it, at least one figure, and a short written discussion tying the result to a specific result from the lectures. Five-minute presentation: one slide of setup, one figure, one sentence of “what surprised me.”

Track A — Scale & fine-tune

For: “I want it to actually work better / do something specific.”

Choose a direction:

Grow it. Push $d$ , $L$ , $T_{\max}$ , and the dataset as far as your compute allows (Colab GPU strongly advised; consider a bigger corpus — Project Gutenberg, a code dataset, your own writing). Track bits/char and place the run on your Step 7 scaling plot. Does the fitted power law predict where you land?
Fine-tune (transfer). Pre-train on one corpus, then continue training (“fine-tune”) on a small, different one (e.g. general English → a single author). Show, quantitatively, adaptation: val loss on the target domain before vs after, and a qualitative style shift in samples. This is the base-model → specialization move of Lecture 8, in miniature.
LoRA. Implement low-rank adaptation (Hu et al. 2021): freeze the pretrained weights, add trainable $BA$ with $B \in \mathbb R^{d\times r}, A \in \mathbb R^{r\times d}$ , $r \in \{2,4,8\}$ , to the attention projections. Match full fine-tuning’s target-domain loss while training $< 5\%$ of the parameters; plot quality vs $r$ . State the hypothesis LoRA rests on (the update is low-rank) and whether your numbers support it.

Track B — Alignment / preference optimization

For: “I want to understand RLHF/DPO as mathematics.”

Your model is a base model — it continues text, it doesn’t follow instructions. Do a scaled-down version of the alignment pipeline on a toy but well-defined preference:

Define a programmatic reward $r(x)$ on generated text — e.g. “+1 per line that scans as iambic,” or “prefers dialogue over stage directions,” or “avoids a banned word.” (Programmatic, so you need no human labels.)
Derive (this is the mathematical heart — do it in the notebook) the KL-regularized objective $\max_\pi \mathbb E_{\pi}[r] - \beta\,\mathrm{KL}(\pi \,\|\, \pi_{\text{ref}})$ and its closed-form optimum $\pi^*(y) \propto \pi_{\text{ref}}(y)\,e^{r(y)/\beta}$ (Lecture 8).
Implement either rejection sampling / best-of- $n$ against $\pi^*$ (easy, and already reveals the alignment tax), or the DPO loss on pairs $(y_w, y_l)$ you generate and rank by $r$ . Show samples shifting toward high reward as $\beta$ decreases, and measure the KL from the base model — exhibit the reward-vs-KL trade-off curve. Where does the model start “hacking” your reward? (It will; describe the exploit.)

Track C — Mechanistic interpretability

For: “I want to know what the weights I trained actually compute.”

Reverse-engineer a piece of your own trained model, using the transformer-circuits framework (Elhage et al. 2021; Olsson et al. 2022):

Attention atlas. For each head, plot its attention pattern averaged over many sequences and classify it: previous-token, positional, delimiter-attending, content-based. Which of your $L \times H$ heads are interpretable?
Hunt for an induction head. Construct a repeated random sequence [A B C … A B C …] and find heads whose attention at the second B points back to the token after the first occurrence of the current token (the [A] → B copy rule). Measure the model’s in-context copying: does loss on the repeated half drop below the fresh half? That gap is in-context learning, in your 0.8M-parameter model.
Ablate and confirm. Zero out the candidate head’s contribution to the residual stream (or its OV circuit) and show the copying ability drops. Causal confirmation, not just correlation — the standard of evidence Lecture 8 holds interpretability to.

Optional deeper cut: extract a head’s QK and OV circuits as matrices (Step 5 “going further”), and interpret the induction head as the composition of a previous-token head (earlier layer) feeding it — the two-head induction circuit.

Suggested rubric (for the showcase)


A clear, answerable question	25%
Correct experiment + at least one figure	35%
Mathematical connection to a lecture result, stated precisely	25%
Presentation (5 min, one figure, one surprise)	15%

Where to go after the course

nanoGPT / modded-nanoGPT — take your understanding to a model that trains on real hardware at real scale.
The Transformer Circuits thread — if Track C hooked you, this is the research frontier; much of it is open problems accessible from exactly what you now know.
Deep learning theory (Telgarsky’s notes; Roberts–Yaida–Hanin) — if you want the theorems the empirical parts of this course left open.
Your (step, bits/char) table, from Step 1 to here, is the story of the course in one figure. Put it on your last slide.

Catch-up

Load solutions/step-07.ipynb’s checkpoint and tokenizer; every track begins from a trained model, so even a late joiner can do a full capstone from Lecture 8 alone.