Vision Meets Transformers
Vision Transformers
9 min+90 XPStep 1/10
Slice a photo into tokens
The Vision Transformer (ViT) had a scandalously simple idea: cut the image into fixed patches — say 16×16 pixels each — flatten every patch into a vector, add position information, and feed the sequence to a completely standard transformer.
To the model, a photograph is just a sequence of tokens, no different in form from a sentence. The same attention mechanism that relates words relates patches of pixels.
insight
ViT barely modified the language transformer at all. The bet was that attention is a general-purpose way to relate pieces of input — and it paid off.