The Landmark Papers
Attention Is All You Need (2017)
Throwing out recurrence
Attention Is All You Need (Vaswani et al., 2017) discarded both recurrence and convolution entirely, keeping only attention — a mechanism where every token directly weighs its relationship to every other token.
What it showed: a pure-attention architecture — the transformer — could beat the best recurrent networks on machine translation while training dramatically faster, because all positions are processed in parallel rather than step by step.
Why it mattered: RNNs had a hard bottleneck — sequential processing can't be parallelized, and gradients degrade over long chains. Attention removed both in a single move.
analogy
RNNs read a sentence like a novel — word by word, remembering. Attention reads it like a diagram — everything visible at once, with lines drawn between related words.