Downhill by Feel

Gradient descent needs no calculus to perform: nudge a knob, measure the error's change, step downhill, repeat.

A one-knob predictor guesses y as w times x; its pile is the pairs one-two and two-four; its error works out to five times the square of w minus two. Start ignorant at w equals zero: altitude 20. Feel the slope: at w equals 0.1 the error reads 18.05; at minus 0.1, 22.05 — downhill is toward larger w. Stride half a unit and log the walk: 20, then 11.25, then 5, then 1.25, then 0 — the valley floor, at w equals two, the true rule behind the pile, found by nothing but nudge-and-measure. Now the stride lesson: from zero, a stride of five lands at w equals five, altitude 45 — uphill, the valley overshot entirely.