TPUs & the Accelerator Landscape

Google's TPU took the GPU's insight further: if AI is just matrix multiplication, build a chip that is a matrix multiplier. A systolic array is a grid — 128×128 or 256×256 — of multiply-accumulate units. Weights are loaded into the grid; activations pulse through rhythmically, each cell multiplying and passing its partial sum to the next.

No instruction fetch per operation, no cache thrash — data flows like blood through a heart (hence 'systolic'). A TPU pod links thousands of these chips with a custom interconnect, and Google trained most of its flagship models on them.