Tokenizing Pixels & Frames

Transformers want sequences; a 224×224 image is 150,528 values — as raw pixels, attention is dead on arrival. ViT's move: slice the image into 16×16 patches, flatten each, project linearly — 196 tokens, each a learned 'word' made of texture and edges.

What patchification quietly decided:

• Locality is free — within-patch structure is handled by the linear projection; attention only models between-patch relations. Cheap, but the model must learn fine detail through a bottleneck.
• Position is injected, not native — images have 2D geometry; sequences don't. Position embeddings re-teach the transformer where patches live.
• Resolution is a token budget — double resolution = 4× tokens. Every high-res decision is a context-length decision.