section 06 · status: live · 6 entries · updated 2026-08-29
RAG & Embeddings
Models know what they read during training and nothing since. Retrieval fixes that: meaning as vectors, similarity search at scale, the pipeline that feeds documents into context, how to measure whether any of it is working, and the honest comparison with fine-tuning.
live · a query landing in embedding space, pulling neighbors into context
- Embeddings3 minmeaning as a point in space
- Vector search3 minfinding neighbors without checking everyone
- The RAG pipeline3 minlook it up before answering
- Agentic retrieval2 minstop retrieving once; let the model hunt
- Evaluating retrieval3 minmeasure the ceiling before you polish the room
- RAG vs fine-tuning3 minknowledge in context, behavior in weights
check yourselfAnswer before you open
Trying to recall something teaches it better than re-reading does. Have a go, then open the answer.
Your RAG system gives confidently wrong answers. Where do you look first?
Retrieval, not the model. If the answer-bearing chunk never reaches the prompt, a good model produces fluent garbage. Measure it separately: for real questions, how often is the right chunk in the top k? That number is usually the system's ceiling. The RAG pipeline →
Why can't you mix embeddings from two different models in one index?
Vector spaces are not compatible across models. A query embedded with one model scores garbage against documents embedded with another, and nothing errors — quality just quietly craters. Upgrading the embedding model means re-embedding every document. Embeddings →
Your approximate index returns ten plausible neighbors. What is missing?
A recall number. Approximate search never errors; it just omits the right answer sometimes. Sample real queries, compute ground truth by brute force, and measure recall@k — then measure it again after heavy churn. Vector search →
Your retrieval evaluation shows recall@5 of 60%, but end-to-end answer accuracy measures 75%. Is this good news?
No — it is a warning. Retrieval caps grounded answers, so accuracy above recall means the model is answering from what it memorized in pretraining. Those answers are unattributable, silently stale, and absent on the private half of your corpus. Score faithfulness against the retrieved text, not just correctness. Evaluating retrieval →
When is fine-tuning the right answer instead of retrieval?
When you need form, not facts: a house voice, a rigid output shape, a domain vocabulary, or lower cost and latency from a smaller model. Facts that change, need citation, or must be revoked belong in retrieval — updating an index is a deploy, retraining is a project. RAG vs fine-tuning →