Inside the Vector DB

Brute-force nearest neighbor scores every vector in the index: 100M vectors × 768 dims, every single query. That's a full-time compute job for one request.

Approximate nearest neighbor (ANN) indexes trade a little recall for orders of magnitude of speed — return 95% of the true neighbors in 1% of the time. The whole vector-DB field is variations on one bargain: give up a little correctness, get back a lot of speed.