Applied & Classical ML · entry 04/06 · 3 min read

Forecasting

Time series break nearly every assumption ordinary supervised learning rests on — and the baseline you must beat is embarrassingly simple.

What makes time different

Ordinary supervised learning assumes rows are independent draws from one distribution. A time series is neither: today depends on yesterday, the distribution drifts by design, and the thing you are predicting has not happened yet to anyone. That breaks the standard toolkit in specific ways — random train/test splits become leakage, cross-validation should roll forward rather than shuffle, and any feature computed over a window has to use only data available before the forecast origin.

Beat the naive baseline first

The discipline's most useful habit: before anything else, score the simplest possible forecasts. Last observed value. Last year's same week. A seasonal average. A rolling mean. On a startling share of real problems these are hard to beat. The M-competitions found for decades that combinations of simple statistical methods outperformed elaborate ones — until M5, run on roughly forty thousand related retail series, where boosted-tree methods swept the leaderboard and beat the best statistical benchmark by more than twenty percent. The older lesson survives in a narrower and still-useful form: on a single series with a short history, simple is very hard to beat. If your model does not clear "same as last week," it has not earned its deployment.

The three families

Statistical — exponential smoothing, ARIMA and their seasonal variants — fit one series at a time, are fast, and are genuinely hard to beat on short series. Global machine learning reframes forecasting as tabular regression: build lag features, calendar features, and rolling statistics, then hand them to a gradient-boosted tree trained across all series at once, which lets a thin series borrow strength from thick ones. Deep models — sequence architectures trained across a whole panel — pay off with many series, long histories and shared structure. Pretrained forecasting foundation models invert that bet: they aim at the series you cannot train on, forecasting a new one zero-shot, and whether they beat a tuned baseline there is still being argued out.

What people actually need

A point forecast is rarely the deliverable. Inventory decisions need a quantile ("stock to cover 95% of weeks"), staffing needs a range, and capacity planning needs to know how wrong the number can be. Predict intervals and score two things at once: whether they contain the truth as often as they claim — which is calibration, applied to time — and how narrow they are while doing it, since coverage alone is satisfied by an interval wide enough to be useless. Pinball loss reads both halves. For point error use a scale-free measure that survives zeros, such as MASE or RMSSE, so a series of millions and a series of dozens can be compared.

Failure mode

Forecasting a series that a decision will change. Demand forecasts drive pricing and stock, which drive demand; a promotion forecast informs the promotion. Once the forecast is an input to the world it describes, accuracy against what actually happened stops being the right score, and chasing it teaches the model the effects of your own past interventions. Model the intervention explicitly, or forecast the counterfactual and treat the decision separately.