glossary · 84 terms · filtering is local — nothing leaves the page

Say that again.

Every term this site uses, defined in a sentence or two, each linked to the entry where it is actually explained. Meant to be read cold — if you landed here from a search, start with the word you came for.

• Adapter (LoRA)A small set of extra weights trained alongside a frozen model so it learns a new behavior without retraining the whole thing. QLoRA adds 4-bit base weights so a serious model tunes on one GPU.
• AgentA model given tools, memory and a goal, run in a loop until the goal is met or something stops it. The model decides which steps to take; your code executes them.
• AlignmentThe engineering problem of making a system pursue what you meant rather than what the training signal literally rewarded.
• Anomaly detectionFlagging what does not look like the rest of the data, usually without labels. Note that merely unusual is not the same as the thing you care about.
• AttentionThe mechanism that lets each token weigh every earlier token by learned relevance, and the reason transformers replaced everything before them.
• BackpropagationThe efficient sweep backward through a network that computes, for every weight, which way to nudge it to reduce the error.
• Base modelThe model straight out of pretraining: enormous knowledge, no manners. It completes text rather than answering questions.
• Benchmark contaminationWhen test questions leak into training data, so a model 'generalizes' brilliantly to material it has already read.
• Bias (fairness sense)Systematic disparity in a system's behavior across groups, arriving from data, labels, sampling or a proxy target — usually all four.
• CalibrationWhether a model's confidence means what it says: of everything scored 0.7, roughly 70% should turn out positive.
• Chain of thoughtReasoning written out in intermediate tokens before the answer. It buys accuracy by spending computation, since each token gets a fixed amount.
• ChunkingSplitting documents into passages small enough to embed and retrieve. A design decision, not plumbing — chunk boundaries decide what can be found.
• Classifier-free guidanceThe knob in image generation that controls how literally the model follows the prompt, by exaggerating the difference the prompt makes.
• Collaborative filteringRecommending from behavior alone — people who liked these also liked that. Powerful, and silent about any item nobody has touched yet.
• Computer useAn agent operating software through screenshots, mouse and keyboard — the universal integration for applications that have no API.
• Constitutional AITraining a model against a written set of principles: it critiques and revises its own outputs, then trains on the result.
• Context engineeringDeciding what earns space in a model's limited window each turn — compaction, tool-result hygiene, externalized memory, cache discipline.
• Context windowEverything a model can see at once, measured in tokens. Outside it, the model has no memory whatsoever.
• ConvolutionA small learned filter slid across an image, firing wherever its pattern appears — the operation that made computer vision work.
• Cosine similarityThe angle between two vectors, used to score how close two embeddings are. The ranking matters; the absolute number means little.
• DiarizationWorking out who spoke when — separating a recording into speakers before or alongside transcribing the words.
• DiffusionGenerating an image by starting from pure noise and repeatedly removing a little of it, guided by a prompt.
• DistillationTraining a small model on a large one's outputs — the workhorse of cost reduction, and how most small reasoning models are actually made.
• EmbeddingA list of numbers representing a piece of text or an image, arranged so that closeness in the space tracks similarity in meaning.
• Epistemic uncertaintyThe model's own ignorance — high where training data was thin. Unlike noise in the world, more data fixes it, and it is what should trigger abstention.
• EvalA graded set of real cases that defines what 'working' means, catches regressions, and encodes the failures you promised not to repeat.
• Few-shot promptingShowing a model two or three worked examples in the prompt so it infers the task, the format and the level of detail — no training required.
• Fine-tuningAdditional training on your own examples to change how a model behaves. Good for form and voice; bad at installing facts.
• Gaussian splattingRepresenting a captured 3D scene as millions of translucent blobs that render in real time — how phone scans became photoreal.
• GeneralizationWhether a model works on data it has not seen. The only number that matters, because production is new data by definition.
• Gradient boostingFit a shallow tree, fit a second tree to what the first got wrong, add them, repeat. Still the default on tabular data, and still hard to beat.
• Gradient descentThe training loop itself: measure the error, work out which way each weight should move, take a small step, repeat.
• GroundingPutting the truth in the model's context — retrieved documents, tool results — and asking it to answer from that material with citations.
• HallucinationFluent, confident output that is not true. A consequence of training for plausible continuations, not a malfunction.
• HNSWThe dominant approximate nearest-neighbor index: a navigable graph with sparse 'highway' layers, giving roughly logarithmic search for a little lost recall.
• Hybrid retrievalRunning keyword search alongside vector search and merging the results, because vectors fumble part numbers and keywords catch them.
• In-context learningA model picking up a task from examples in the prompt, with no weights changed — the capability that made prompting a discipline.
• InferenceRunning a trained model to get an answer, as opposed to training it. Where per-token costs and latency live.
• InterpretabilityReverse-engineering what a model's weights actually compute — features, circuits — so that trust can rest on mechanism rather than test coverage.
• JailbreakTalking a model past its trained refusals with role-play, obfuscation or gradual escalation.
• KV cacheStored attention state for every token already processed. It is what makes generation fast and what limits how many conversations a GPU can hold.
• LeakageInformation in your training features that would not be available, with those values, when the prediction is actually made. It produces validation scores that feel like a breakthrough and collapse on day one.
• Lethal trifectaUntrusted input, access to private data, and the ability to send data out. Any two are usually fine; all three in one agent is the shape most real exfiltration takes.
• LLM-as-judgeUsing one model to grade another's output. Cheap and scalable, and it inherits the judge's biases, so it has to be calibrated against human labels before the number means anything.
• Loss functionA single number scoring how wrong the model just was. It is the only voice the data has, and the model optimizes exactly what it says.
• MCPModel Context Protocol: one standard interface between models and tools, so N models and M tools need N+M integrations instead of N×M.
• Mixture of expertsAn architecture where a router sends each token to a few of many expert sub-networks, so total parameters can be huge while the compute per token stays small. Memory cost does not: all the weights still have to be somewhere.
• Model cardDocumentation of what a model does, what it was trained on and where it fails — research etiquette that regulation is turning into an expectation.
• Off-policy evaluationEstimating how a proposed policy would have performed, using a log of decisions a different policy made. Trustworthy in proportion to how much the old policy explored.
• Operational design domainThe conditions a system is certified to work in — the roads, weather, and speeds inside which its claims hold, and outside which they do not.
• OverfittingLearning the training data's noise rather than its pattern: perfect on what it has seen, useless on what it has not.
• PPMIPositive pointwise mutual information — weighting word co-occurrence counts by how surprising each pairing is. The basis of count-based embeddings.
• Prefix cachingReusing the computed state of an identical prompt prefix across requests, which providers discount steeply — and why stable prompt ordering is a cost feature.
• PretrainingThe first and most expensive training stage: predict the next token across a filtered slice of the internet, for months, on thousands of GPUs.
• Prompt injectionInstructions smuggled into content a model processes — a webpage, an email, a document — which it may follow as if you had written them.
• QuantizationStoring weights at lower precision (often 4-bit) to cut memory and bandwidth, with a small accuracy cost that must be measured, not assumed.
• RAGRetrieval-augmented generation: find the passages relevant to a question, put them in the context, and have the model answer from that evidence.
• Reasoning modelA model trained to think at length before answering, trading serving compute for accuracy on problems that reward deliberation.
• Recall@kOf the truly nearest items, what fraction did the index actually return. The number that quietly decides whether a retrieval system works.
• RerankingA second-pass model that reads query and passage together to order candidates far better than raw vector distance. Retrieve fifty, keep five.
• Reward hackingAn optimizer satisfying the reward you wrote rather than the goal you had — the boat that spins through checkpoints instead of finishing the race.
• RLHFReinforcement learning from human feedback: people pick the better of two answers, a reward model learns their preference, the model optimizes against it under a leash.
• RLVRReinforcement learning with verifiable rewards: train against problems whose answers can be checked mechanically, which is what taught models to reason at length.
• Scaling lawsThe observation that loss falls on a smooth, predictable curve as parameters, data and compute grow — smooth enough to budget against.
• Self-supervised learningHiding part of the data and training the model to predict it, so every document becomes its own answer key. The trick that unlocked modern AI.
• Shortcut learningA model scoring well by latching onto an incidental correlate of the answer rather than the thing you meant it to learn.
• Sim-to-realTraining a robot policy in simulation and transferring it to hardware, usually by randomizing the simulation enough that reality looks like one more variation.
• SoftmaxThe function that turns arbitrary scores into probabilities that sum to one. It is how attention weights and next-token odds are produced.
• Sparse autoencoderA wide, sparsely-activating layer trained to reconstruct a model's internal state, whose units line up with human-legible features better than raw neurons do.
• Speculative decodingA small draft model proposes several tokens and the big model verifies them in one pass — same output distribution, fewer expensive steps.
• Structured outputConstraining a model to emit valid JSON against a schema, so its output can feed a program rather than a person.
• SuperpositionModels packing more features than they have dimensions, which is why single neurons respond to several unrelated things.
• TemperatureThe sampling knob: low makes the model pick its safest word every time, high spreads probability into the long tail.
• TokenThe unit a model actually reads and is billed in — a word, a word piece, or punctuation, drawn from a fixed vocabulary.
• TokenizerThe frozen vocabulary welded to a model before training, built by merging the most frequent character pairs in its corpus.
• Tool callingA model emitting a structured request — a name and typed arguments — that your runtime executes. The model itself never runs anything.
• TransformerThe architecture behind modern AI: stacked blocks of attention plus small feed-forward networks, predicting one token at a time.
• Two-tower modelA retrieval architecture that embeds users and items into one shared space so the best candidates are a nearest-neighbor lookup away.
• UnlearningRemoving one person's or one document's influence from a trained model after the fact. Still largely an open problem; retraining is the honest method.
• Vector databaseStorage plus an index for embeddings. Worth its complexity at very large scale; below that, your existing database probably suffices.
• Vision-language modelA model that takes images and text together — reading a screenshot, answering questions about a photo, grounding words to pixels.
• WatermarkingEmbedding an imperceptible signal in generated media at creation. Present proves a lot; absent proves nothing.
• Word error rateThe standard speech-recognition score: insertions plus deletions plus substitutions, over the number of words actually spoken.
• World modelA model that predicts the next frame given recent frames and your actions — an explorable environment generated on the fly.

No term matches — try fewer letters, or the full index.