Mechanistic Interpretability

Every technique so far tests what a model does. Mechanistic interpretability asks what a model computes: given the weights, can we reverse-engineer the algorithms the network is actually running?

Two primitives organize the work. A feature is a meaningful direction in activation space — 'this pattern of neuron firings encodes is-a-question'. A circuit is a set of weights connecting features into an algorithm — 'head A detects a repeated token, head B attends to what followed it last time'. The most famous circuit, induction heads, is literally how transformers do in-context learning: find earlier occurrences of this pattern, predict what came next.