Attention Is All You Need

Attention lets each token ask: which other tokens matter for understanding me? In 'the animal didn't cross the street because it was tired', attention connects 'it' strongly to 'animal'.

Each token produces a query, key, and value vector. Attention scores = how well queries match keys. High scores = information flows. Do this for all tokens, in parallel, across many 'heads' — that's the transformer.