Why Vanilla Nets Fail on Sequences

A feed-forward network sees a fixed-size, unordered input. 'The dog bit the man' and 'The man bit the dog' contain identical words — to a vanilla net, they're identical inputs. Word order, which carries the actual meaning, is invisible.

Two more problems: variable length — a tweet is 20 words, a novel is 100,000, and no single input size fits both — and no memory: each input is processed from scratch, so there's no way to remember that 'she' refers to 'Alice' from three sentences ago.