Machine Learning Foundations · entry 03/05

The kinds of learning

Supervised, unsupervised, self-supervised, reinforcement: the real difference is where the training signal comes from and what it costs.

Supervised: paid answers

Inputs paired with correct outputs — photos with labels, emails marked spam. Powerful, direct, and bottlenecked by the price of human answers: labeling a million examples is a payroll, and the model can never exceed the taste of its labelers. Most production ML of the 2010s lived here.

Unsupervised: structure for free

No answers, just data — find the structure. Cluster customers, detect the odd one out, compress dimensions. Cheap signal, vague target: useful mostly as a lens on data rather than a way to perform a task.

Self-supervised: the data grades itself

The trick that unlocked modern AI. Hide part of the data, train the model to predict it: the next word of a sentence, the missing patch of an image. Every document on the internet becomes its own answer key — supervised-strength signal at unsupervised prices. Large language models are self-supervised learners first and everything else second.

Reinforcement: graded by consequences

No correct answer per step — just reward when things go well. The model acts, the world (or a judge) scores the outcome, credit flows back through the choices. It's how systems learn games, robot control, and — as RLHF — how raw text predictors get shaped into assistants. Expensive, unstable, and the only option when the answer is a behavior rather than a label.

In practice: stacked

A modern assistant uses all of them — self-supervised pretraining for knowledge, supervised examples for format and skills, reinforcement for judgment. The stack matters because each stage's signal papers over the previous stage's gaps.

Failure mode

Forgetting the signal's origin. Every learned behavior traces to who provided the answers: internet text (with its myths), labelers (with their guidelines), reward models (with their exploitable blind spots). When a model does something strange, the first question is never "why is it broken" — it's "what signal made this the winning move."