Attention

Attention rewrites every word's row as a weighted average of the other rows in the passage — with the weights computed from how well the rows match, so each word assembles its own context.

Read "the melon that the farmer thumped was ripe" and ask what "was ripe" must attend to: melon, not farmer. Attention's pair-matching, tuned by descent until prediction succeeds, learns to weight exactly so — the melon row pours heavily into the rewrite, the farmer row barely at all. And the one-row problem dissolves: after an attention layer, "bank" in a river sentence and "bank" in a loan sentence have been rewritten by different company, which repairs the flaw video 0510 exposed. The weighted average is Unit 6's oldest tool; the novelty is that the weights are not knobs set once by training but numbers produced on the fly, from the words themselves, at every reading.