section 01 · status: live · 10 entries · updated 2026-08-29
Machine Learning Foundations
Everything else on this site compiles down to this section: what a network is, how training moves it, where the data comes from, and the whole discipline of deciding whether the result works. No mathematics beyond arithmetic — but no hand-waving either.
live · gradient descent feeling its way down a loss curve
- What a neural network is2 minweighted sums, bent, stacked deep
- How models learn2 minwrongness, measured, then walked downhill
- The kinds of learning2 minwho provides the answers — and at what price
- Where data comes from3 minthe model is the dataset, rearranged
- Overfitting & generalization2 minmemorizing the test is not passing it
- Models grading models3 minthe eval that scales, and the biases it brings
- Learning by building3 minreading gives you vocabulary; building gives you judgment
- Keeping up3 mina field that outruns its own textbooks
- Calibration & uncertainty3 mina number that means what it says
- Evaluating models2 minthe score is a proxy; act like it
check yourselfAnswer before you open
Trying to recall something teaches it better than re-reading does. Have a go, then open the answer.
A model scores 99.7% accuracy detecting fraud. Why might that be worthless?
Because 0.3% of transactions are fraud, so a model that flags nothing scores 99.7%. Accuracy under class imbalance is a compliment, not a measurement — you need precision and recall, and a decision about which error costs more. Evaluating models →
Your training loss keeps falling while validation loss starts rising. What is happening, and what are your options?
The model has run out of pattern and started memorizing noise — overfitting. Levers: more data, a smaller model, early stopping, regularization, augmentation. The gap between the two curves is the thing that matters, not either curve alone. Overfitting & generalization →
A model outputs 0.9 for a case. What has it actually promised you?
Nothing, unless it is calibrated. Calibration means that of everything scored 0.9, about 90% turn out positive — check it with a reliability diagram before any score becomes a threshold, and re-derive that threshold if you rescale. Calibration & uncertainty →
You have 40,000 unlabeled support tickets and budget to label 500. What do you do?
Label the 500 as an evaluation set, not a training set. Labels are the scarce resource, so spend them where they compound: a graded set defines what working means and catches every future regression, while 500 training examples train almost nothing. Then use the unlabeled pile however the method allows — few-shot prompting, clustering, self-supervised pretraining. Where data comes from →
Your LLM judge scores the new system 8.4 versus 7.1 for the old one. What has to be true before that means anything?
That the judge agrees with human graders on a sample you checked. Judges have known biases — longer answers, first position, a preference for their own family's output — so an uncalibrated judge is a random number generator with good manners. Also: never optimize against the judge you validate with. Models grading models →
What does a transformer actually compute at the bottom?
Weighted sums and a bend. Every unit multiplies its inputs by learned weights, adds them up, adds a bias, and passes the result through a non-linear function — that bend is the whole reason stacking layers buys anything, because without it a hundred layers collapse into one. What a neural network is →