Q: How do I evaluate agentic workflows?
We recommend evaluating agentic workflows in two phases:
1. End-to-end task success. Treat the agent as a black box and decide whether it met the user’s goal. Define a precise success rule per task and measure it with human review or validated LLM judges. Record the first upstream failure during error analysis.
Once error analysis reveals which workflows fail most often, move to step-level diagnostics to understand why they’re failing.
2. Step-level diagnostics. After you log the system’s traces, you can score individual components such as:
- Tool choice: check whether the agent selected the appropriate tool.
- Parameter extraction: check whether the inputs were complete and well-formed.
- Error handling: check how the agent handled empty results or API failures.
- Context retention: check whether the agent preserved earlier constraints.
- Efficiency: count the steps, seconds, and tokens spent.
- Goal checkpoints: verify key milestones in long workflows.
How do I test tool calls?
Test the tool name, arguments, result, and resulting state as separate checks. Use code assertions when the expected behavior is objective. For example, verify that the agent selected cancel_order, passed the correct order ID, received a successful response, and changed the order status before it told the user that cancellation succeeded.
Also test authorization and preconditions. A valid tool call can still be wrong if the user did not approve the action or the system skipped a required check.
Example: “Find Berkeley homes under $1M and schedule viewings” breaks into: parameters extracted correctly, relevant listings retrieved, availability checked, and calendar invites sent. Each checkpoint can pass or fail independently, making debugging tractable.
Use transition failure matrices to understand error patterns. Create a matrix where rows represent the last successful state and columns represent where the first failure occurred. This is a great way to understand where the most failures occur.
Transition matrices show where failures cluster. In this example, GenSQL → ExecSQL transitions cause 12 failures while DecideTool → PlanCal causes only 2. The counts show where to investigate first. Here is another text-to-SQL example from Bryan Bischof:
In this example, Bryan shows variation in transition matrices across experiments. How you organize your transition matrix depends on the specifics of your application. For example, Bryan’s text-to-SQL agent has an inherent sequential workflow which he exploits for further analytical insight. You can watch his full talk for more details.
Creating Test Cases for Agent Failures
Creating test cases for agent failures follows the same principles as our previous FAQ on debugging multi-turn conversation traces. Reproduce the error with the simplest test that still fails. Use a multi-turn test only when the failure depends on conversation context.
This article is part of our AI Evals FAQ, a collection of common questions (and answers) about LLM evaluation. View all FAQs or return to the homepage.