Agent Evals

A single-agent answer eval asks 'was the output right'. An agent eval must also ask 'was the path sound' — because a lucky trajectory hides a broken agent. The research agent that cited the wrong source but guessed the right answer passed your outcome eval and will fail the next hundred tasks.

Grade both layers:

• Task success — did the environment end in the correct state? For agent tasks this is often checkable in code: the file exists, the DB row is right, the test passes.
• Trajectory quality — did it get there the right way, at the right cost?