History of AI · entry 03/05

The quiet decades

Between the second winter and the deep-learning eruption, AI rebuilt itself as statistics while three ingredients matured in public view — and almost nobody added them up.

Machine learning grows up embarrassed

Through the 1990s and 2000s, the survivors rebranded. "AI" implied overpromise, so the respectable work called itself machine learning and looked like statistics: support vector machines with actual theorems, graphical models with principled uncertainty, boosting, random forests. The field internalized evaluation discipline — held-out test sets, benchmarks, honest baselines — largely as a vaccine against its own history. Useful systems shipped quietly: spam filters, credit scoring, recommendation, speech dictation. Nobody called it AI; it was working too well for that — the field's oldest running joke, that whatever finally works stops counting as artificial intelligence.

The three ingredients, maturing separately

Compute: the gaming industry, chasing prettier explosions, spent two decades building massively parallel processors — and around 2007, researchers noticed GPUs ran neural network math tens of times faster than CPUs. Data: the web accidentally assembled humanity's largest labeled dataset — billions of images with captions, text with structure — and Fei-Fei Li's ImageNet (2009) distilled it into a million labeled images and, crucially, a public competition with a leaderboard nobody could argue with. Algorithms: a small community — Hinton, LeCun, Bengio, Schmidhuber's lab among them — kept improving multi-layer training through the unfashionable years: backpropagation refined, convolutional nets reading checks in production at Bell Labs, LSTMs solving sequence memory. Each ingredient was public. The sum was not obvious. Almost everyone priced neural networks at their 1969 valuation.

The detonation

In 2012, Hinton's students Alex Krizhevsky and Ilya Sutskever trained a convolutional network on two consumer GPUs and entered ImageNet. AlexNet's error rate: 15.3%. The runner-up, representing the entire non-neural computer vision tradition: 26.2%. Fields don't usually produce results that unambiguous. Within three years every serious vision group had converted, speech recognition followed, and the quiet decades' three ingredients became the loudest technology on earth.

Failure mode

Assuming the next paradigm will announce itself. The lesson of 2012 is that revolutions assemble in public from unglamorous parts — a gaming chip, a labeling effort, an algorithm everyone had already dismissed — and the tell isn't a manifesto but a benchmark gap too large to argue with. Watch for ingredients compounding and leaderboards jumping, not for consensus; consensus is a lagging indicator by definition.