Error Taxonomies: Name Your Failures

An error taxonomy is a classification of how the system fails — and the only honest way to build one is to read failures until they stop surprising you. Take a hundred failed outputs/traces, label each by what actually went wrong, and let the categories emerge from the data: 'retrieval miss', 'instruction ignored', 'format violation', 'hallucinated citation', 'wrong tool', 'right answer wrong reason'.

A taxonomy imported from a paper or invented in a meeting classifies the failures you expected, which are rarely the ones you have. Production failures are stranger.