Reading Word by Word: RNNs & LSTMs

The recurrent neural network had a human-shaped idea: read one word at a time and keep a running summary. 'The' → update the hidden state. 'cat' → update again. By 'sat', that vector has absorbed 'the cat'.

Same small network, same weights, every step — like a reader rewriting a one-line summary as they go. Variable-length input, order preserved: on paper, exactly what language needs.