Policy Gradients

Value methods are indirect: learn what actions are worth, then recover a policy by argmax. Why detour? Parametrize the policy itself — π_θ(a|s), a probability distribution over actions — and ask the direct question: which θ maximizes expected return?

REINFORCE, the canonical policy gradient, is almost embarrassingly simple: run an episode, see the return, then raise the probability of every action taken in proportion to it. Good episode → those moves get more likely. Bad episode → less likely. Gradient ascent on reward.