Text as Data
N-grams: The First Language Models
8 min+75 XPStep 1/10
Predict the next word — by counting
An n-gram is just n words in a row: 'the cat' is a bigram, 'sat on the' a trigram. The leap was treating counts as a language model: given 'I want a cup of', what comes next? Tally a corpus and get odds — 'tea' 30%, 'coffee' 25%, 'soup' 10%.
Claude Shannon was playing this game in 1951, asking people to guess the next letter of English text to measure its entropy. By the 1990s, n-gram models ran speech recognition and the first statistical translators.
analogy
Your phone's autocomplete is the great-grandchild of this idea — 'how are' suggesting 'you' is an n-gram table doing its one trick.