The Lab · instrument 01/09 · interactive

Train a neural network

A real multilayer perceptron — forward pass, backpropagation, Adam — training live in your browser. Paint your own data and watch the network fight to fit it.

What's real here

Everything. This is a genuine 2–16–16–2 network with a softmax output and cross-entropy loss, trained by backpropagation with the Adam optimizer, hundreds of steps per second, entirely in this page. The colored wash is the network's current opinion sampled on a grid — violet where it predicts one class, cyan the other, washed out where it isn't sure. The difference between this and a frontier model is mostly count: a few hundred parameters here, hundreds of billions there, the same arithmetic in the loop.

Where to push

Drop the units to 4 on the spiral and watch the network try to bend two lines around a curve it cannot represent — that's underfitting, live. Push noise up and capacity to 16×3 and watch the boundary contort around individual speckles: that shape is overfitting, and you will now recognize it anywhere. Switch tanh for relu and notice the boundary turn polygonal — every architecture choice has a signature. Then set the learning rate to its maximum and watch training thrash without converging; set it to minimum and watch nothing visibly happen. The window between those two is where all of deep learning lives.

Paint against it

Turn paint on and add a cluster of violet points deep in cyan territory. The boundary reaches out to claim them — at some cost to its elegance elsewhere. You are doing manually what every mislabeled example in a real training set does silently: bending the model toward it. A network has no notion of "that label looks wrong"; the loss says fit it, so it fits it.

Failure mode

Trusting the training accuracy in the corner. It measures the points on screen — the training set — and you can drive it to 100% on pure noise with enough capacity. Nothing on this canvas measures how the network would do on points it hasn't seen, which is the only question production ever asks. The picture is a diagnostic, not a verdict; hold out data the model never touches, or the number lies to you sweetly.