Skip to main content
The Suite tab shows what your agent is tested on, and how that was worked out.
Built from your agent
Scenarios come from your agent’s own structure, not a generic checklist.
Aimed where agents break
Hard decisions, failing tools and hand-offs, not every path equally.
Worth trusting
A few minutes here is how you come to trust the results that follow.
The Suite tab of the report

The Suite tab: the paths your agent can take, and the scenarios that walk them.

​
What to look at

​
The flow graph

The map of every path your agent can take: its steps, decisions and tool calls, built from your code during the scan. Select a scenario to light up the route it walks.
  • Test smarter, not more: the suite goes after the routes where agents break, such as hard decisions, tools that can fail and hand-offs between steps.
  • Small suite, real coverage: that’s why a handful of scenarios can still catch what matters.

​
Scenarios

Each card is one situation your agent will face: who the user is, what they want, how they talk, and what the agent must get right.
  • Aimed, not random: each one is built to reach a specific risky route on the flow graph.
  • Real-life twists: a user who changes their mind, a missing detail, a tool that returns something unexpected.
  • Real people, not scripts: the persona talks like a real user would.
Every card says where it came from:

​
Evals

The checks that decide pass or fail. See Reading an eval below.

​
Tested knowledge

If your agent answers from documents, this shows which passages the suite asks about.
  • Checked against the source: questions are planted in the scenarios with their expected answers.
  • Blind spots in view: you can see which documents nobody has asked about yet.

​
Reading an eval

Evals are your definition of “good”. Every conversation in the run is scored against every eval, so they decide what counts as a gap. Each one tells you: Why evals are built this way:
From your own rules
Each eval traces back to your PRD, your words in chat, a tool’s contract or a scenario’s goal.
Severity cuts the noise
Critical means stop the release; advisory is a nudge. Your fix plan is ranked by it.
Every verdict explained
Each pass or fail ties to a written “fails when” condition you can change.
The right judge
A model judge for tone and judgment; code for exact rules like the right tool or price.
Checked before you see them
Evals are tested against the scenarios so they separate good from bad behavior.

​
Ask for changes

Say what you want in chat. For example:
  • “Add a scenario where the customer is angry about a late refund.”
  • “This eval is too strict, the agent may round prices.”
  • “Test what happens when the order lookup fails.”
The suite updates, and the report shows the change.
If something important is missing, say so before the run. It’s quicker to add a scenario now than to re-run later.
For every label in this tab, see Suite and Traces.