Pruning & Distillation

Pruning removes parameters the network doesn't need. The critical split:

• Unstructured — zero individual weights anywhere; produces sparse matrices. High compression ratios on paper, but sparse matrices only run faster on hardware with sparsity support (NVIDIA's 2:4 structured sparsity is the exception — half the weights removed in a regular pattern hardware understands).
• Structured — remove entire channels, attention heads, or layers; the result is a smaller dense model that runs at speed on any hardware. Layer-drop and head-pruning for LLMs are this.

The honest math: 50% unstructured sparsity on unsupported hardware is 0% faster — the bytes still move, just with zeros in them. Structured pruning is what ships.