SVMs: The Widest Street

If two classes are separable, infinitely many lines separate them. A support vector machine chooses the one with the largest margin — the widest empty street between the two neighborhoods.

Only the closest points — the support vectors — touch the curb and decide where the street runs. Every other point could move around freely without shifting the boundary at all.