Activation Functions

Stack linear layers and you get... a line. A hundred layers of pure matrix math collapses into a single matrix — all that depth, zero extra power.

Insert a bend after each layer — an activation function — and each layer can now warp the space the next one reads. Enough bends, and the network can fold space into almost any shape. This is the entire reason deep learning works.