Exploration vs. Exploitation

You're in a new city for a week. Night one you find a decent restaurant — now every night is a choice: exploit — return to what you know is good — or explore — risk a bad meal for the chance of a great one.

Always exploit and you may spend the week at the third-best restaurant in town, never knowing. Always explore and you burn the vacation on bad pasta. There is no principled escape from the trade-off — every RL agent must budget for the unknown, forever.