The Lab · instrument 04/09 · interactive

Drive an attention head

One attention head computing real dot products over your sentence — hover a token and watch where its attention flows, then sharpen the softmax and see selection emerge.

What's real, what's toy

The computation is the genuine article from transformers & attention: each token is projected into a query and a key, every query is dotted against every key, scores are scaled and softmaxed into weights. What's toy: the projections are random, not trained, and each word's embedding is a hash, not a meaning. So the head shows you real mechanics with fake semantics — in a trained model the same wiring, tuned by loss, is what lets "it" find "the ball."

Where to push

Repeated words share an embedding here, so watch "the" light up "the" — similarity in, attention out, no rule required. Raise the position mix and content stops mattering: tokens attend to their neighbors, which is roughly what some real heads specialize in. Now work the sharpness slider — it scales the scores before the softmax. Low, and attention spreads evenly: every token contributes mush. High, and the head commits to one or two tokens: selection. Real transformers sit in between, and the 1/√d scaling in the formula exists precisely to keep them there.

The mask

Toggle the causal mask and watch the right half of the matrix appear and vanish. With it on, no token may look forward — which is what makes generation possible: the model must predict the future, so it is forbidden from reading it. Every autoregressive model you've used lives entirely in the masked lower triangle you're looking at.

Failure mode

Reading attention maps as explanations. You just watched plausible-looking attention patterns fall out of random weights — proof by demonstration that a pretty heatmap needs no meaning behind it. Trained maps show where information could flow, not why the model answered as it did. Treat them as anatomy, never as testimony.