Words Become Geometry

In 1957, linguist J.R. Firth wrote: 'You shall know a word by the company it keeps.' Meet 'shallot' in enough recipes — near 'onion', 'garlic', 'sauté' — and its meaning leaks through without a dictionary.

Word2vec (Google, 2013) industrialized that intuition: train a tiny network to predict a word from its neighbors — or neighbors from the word — and each word ends up a ~300-number vector where context-twins land close. GloVe (Stanford, 2014) reached similar spaces by compressing corpus-wide co-occurrence counts instead. Either way: meaning, as geometry.