Inside the Transformer

The Full Block

9 min+90 XPStep 1/11

Attention is only half the block

A transformer block is a repeatable sandwich:

Multi-head attention — tokens exchange information
Add & norm — residual connection + layer normalization
Feedforward network — each token is processed on its own
Add & norm — once more

Attention mixes information across positions; the feedforward layer transforms each position privately — and it quietly holds most of the block's parameters.

analogy

Attention is the meeting where tokens share notes; the feedforward layer is each token going back to its desk to think alone.