Deep Q-Networks

A Q-table works for a grid world with a hundred states. Atari's state space is every possible frame — astronomically more states than a table could hold, let alone visit enough times to fill.

Deep Q-Networks swap the table for a neural network: pixels in, a Q-value per joystick action out. The network doesn't memorize states — it generalizes. Frames that look similar get similar values, so learning in one state teaches the net about millions of states it has never seen.