Bag of Words & TF-IDF

Bag of words is the crudest useful idea in NLP: represent a document as nothing but word counts. 'great film, great acting' becomes {great: 2, film: 1, acting: 1} — order, grammar, and nuance all discarded.

And yet it works absurdly well. Spam betrays itself in vocabulary — 'FREE', 'winner', 'guaranteed' — no matter where those words sit. In 2002, Paul Graham's essay A Plan for Spam popularized Bayesian filtering on exactly this: count words, let probability separate junk from mail.