The Lab · instrument 05/09 · interactive

Sample from a language model

A character-level language model learned from a paragraph, generating live — with the same temperature and top-k controls production samplers use, and every failure visible.

What's real here

This is a complete language model — a table of "after these characters, which character came next" learned from the corpus — and a real sampler on top: counts scaled by temperature, optionally trimmed to the top-k, then drawn from. Production models replace the counting with a transformer's learned probabilities and the characters with tokens; the knobs you're holding are otherwise the same knobs.

Where to push

Set temperature to 0.1 and generate: the model plays only its safest notes, and the prose goes repetitive and cautious — cold sampling is how you get a model that says the same thing five ways. Push toward 2.0 and the long tail gets its vote: inventive for a sentence, then word salad. Now hold temperature at 1 and walk the order slider. Order 2 produces alphabet soup that is somehow still English-shaped; order 7 stitches together whole phrases of the corpus verbatim — fluency by memorization, the small-data version of a big model reciting its training set.

The uncomfortable lesson

Nothing in this machine knows what a whale or a marriage is. It has counts. Yet it produces text with the rhythm and vocabulary of its author, and at order 4 it can fool a glance. Hold onto that feeling the next time fluent prose arrives from a model: fluency is the cheapest thing a language model manufactures, which is exactly why hallucination reads so smoothly.

Failure mode

Tuning temperature by vibes and leaving it. The right setting is a property of the task, not the model: extraction and code want cold, brainstorming wants warm, and the same knob that makes a chatbot lively makes a JSON emitter invent fields. Set it per use case, write it down, and retest when you change models — defaults differ, and "creative" is not a compliment in a parser.