> ## Documentation Index > Fetch the complete documentation index at: https://docs.plurai.ai/llms.txt > Use this file to discover all available pages before exploring further. # Simulation run results > Read the attempt grid, pass rates and consistency of a Run, and open any conversation at the turn where it broke. **Simulation run results** shows how each scenario played out and how the evals scored it. It is on the **Run** stage (`#run`) and fills in while [the Run](https://docs.plurai.ai/guides/run-the-simulation.md) plays. *The simulation run: how each scenario played out, scored by the Evals.* ## The headline The top of the section reads, for example, **5 of 7 scenarios passed**, with how many times each scenario was played (**plays per scenario**). The final Run plays every scenario 3 times, as 3 independent attempts. A verify Run after fixes, and the Runs inside the suite build, play each scenario once. ### Pick a Run When the project has more than one Run, a switcher picks which one the section shows: | Label | Meaning | | - | - | | **Playing now** | The Run in progress. | | **This Run** | The latest finished Run. | | **Earlier Run** · date | A previous Run. Expand **earlier Runs** to see them, **Hide earlier Runs** to fold them. | | final / repeat / after fixes | What the Run was for: the scored Run, a repeat of it, or the Run that checks your fixes. | ## The attempt grid *An Attempt is one playthrough of a scenario: one conversation, one square.* ![The attempt grid](https://mintcdn.com/plurai/V3XeGRFO5iyJejs5/images/report/run.png?fit=max\&auto=format\&n=V3XeGRFO5iyJejs5\&q=85\&s=49c4f85cfc5dd575d58723e1a188813f) The attempt grid: every scenario played three times, with the evals that failed. Each row is a scenario. Each square is one attempt at it. Hover or focus a square to read its outcome. Select a scored square to open the conversation at the turn it broke. While the Run plays, *squares land as each conversation finishes; outcomes follow once the Run is scored.* A square first shows **Pending**, then **Finished**, then its outcome. | Square | Meaning | | - | - | | **Passed** | The conversation met its goal and no eval that counts failed. | | **Failed** | The goal was not met, or an eval failed. A crash in your agent counts as a fail. | | **Inconclusive** | The evals could not reach a verdict on this conversation. | | **Not applicable** | No eval applied to this conversation. | | **Invalid** | The simulated user could not play the scenario as written, so it is not scored. | | **Lost** | The conversation did not complete because of the simulation's setup or infrastructure, not your agent. Re-run it. | | **Withdrawn** | The attempt was taken out of the score. | | **Transcript not available** | The attempt was scored, but its conversation was withheld or trimmed to fit the page. | If the Run stops early, the grid says *The Run stopped here. Finished conversations keep their square; the rest never played.* The summary under the grid counts what didn't count, for example *21 conversations, 1 withdrawn, 2 lost*. ### Consistency With more than one attempt per scenario, each scenario gets a consistency label: | Label | Meaning | What to do | | - | - | - | | **Consistent pass** | Every attempt passed. | Nothing. | | **Consistent fail** | Every attempt failed. | A reliable failure. Start here. | | **Inconsistent** | Some attempts passed and some failed. | The agent is flaky on this scenario. Open a failing attempt and compare it with a passing one. | | **Insufficient** | Too few attempts were scored to tell. | Re-run the scenario. | | **Not applicable** | No eval applied. | Check the scenario's evals in [the suite](https://docs.plurai.ai/report/suite.md#evals). | > [!TIP] > > A single Run can't tell a real fix from chance. When the page says *Run-to-run noise is unknown, so one > Run can't tell a fix from chance*, treat a pass after a fix as a weak signal and run the suite again. ## Evals The **Evals** table below the grid scores each eval across the Run. | Column | Meaning | | - | - | | **Eval** | The eval's name. Select it to see **Scenarios that failed this Eval**. | | **Pass rate** | The share of conversations that passed this eval, for example `86%`. | | **Conversations failed** | How many conversations failed it, out of how many it scored, for example `3 / 21`. | The header reads, for example, *21 conversations, each scenario played 3 times · 2 Evals failed at least once*. Evals that never failed are folded: use **Show N Evals that never failed** to list them. The table also notes evals that applied to no conversation, and evals that reached no verdict, in this Run. ## The conversation drawer Select a square, a failing eval row or a gap's scenario to open the conversation. It opens at the turn where the conversation broke. Link to it with `#run.drawer-.run-.attempt-.turn-`. ![The conversation drawer](https://mintcdn.com/plurai/V3XeGRFO5iyJejs5/images/report/replay.png?fit=max\&auto=format\&n=V3XeGRFO5iyJejs5\&q=85\&s=f6a7f7ff78527913fcd0028ac4136465) The conversation drawer: the full conversation, with every tool your agent ran. ### The header The header shows the attempt's outcome, its consistency label and the tally, for example **Failed** · Attempt 2 · Inconsistent · 1 of 3 attempts passed. A scenario played once reads **Played once**. Switch attempts at the top. **Back to the first failing attempt** returns to the first attempt that failed. ### What broke **What broke** lists each eval this attempt failed, with the turn it failed on and the gap it belongs to, if any, with that gap's severity. Select the gap to open it in [Gaps and fixes](https://docs.plurai.ai/report/gaps-and-fixes.md). When nothing failed, it says **No Eval failed on this attempt.** ### The conversation Each turn shows the **User** (the simulated user) and the **Agent**. Failed turns are marked, for example `turn 4 · failed`. Turns without failures are folded: use **show N turns without failures** to read them. For each failure in a turn: | Detail | Meaning | | - | - | | **The line it flagged** | The exact words the eval objected to. | | **Reasoning** | Why the judge failed it. | | confidence | How sure the judge is, for example `0.92 confidence`. | | agreement | When an eval is scored more than once, how many scorings agree, for example `2 of 3 scorings agree`. | Tool calls appear inline as **The agent ran** ``, with its arguments and **Output of** ``. Each call says who answered it: **served by the twin** (the mock environment) or **your system**. A call that errored shows **failed**. The agent's own actions show as events: **Handed the conversation off**, **Conversation closed by the agent**, or **Waiting for the user's decision**. ### Ended early A conversation that stopped before its natural end says why: | Reason | Meaning | | - | - | | *Ended early: the turn limit was reached* | The conversation hit the scenario's maximum number of turns. | | *Ended early: the time budget ran out* | The scenario ran past its time budget. It is graded up to the turn it reached and counts as neither passed nor failed. | | *Ended early: the simulated user gave up* | The persona stopped trying, usually because the agent wasn't getting anywhere. | | *Ended early: the simulated user ran out of script* | The persona finished every beat it had to play. | | *Ended early: the conversation was handed off* | The agent handed the conversation to a human or another system. | | *Ended early: the agent closed the conversation* | The agent ended the conversation itself. | | *Ended early: a policy stopped the conversation* | An eval that stops the conversation on a breach fired. | | *Ended early: the agent ran out of steps* | The agent hit its own step or recursion limit within one turn. | | *Ended early: the replay was cut short* | A replayed production conversation could not be replayed to its end. | ## Run integrity Some facts change how the numbers must be read: trials that could not run on our side, scenarios cut by the time budget, or a flow removed because it can't finish within that budget. When one applies, the Run's chat summary carries one **Run integrity** line, for example: ```text theme={null} Run integrity — 2 of 27 trials could not run on our side, rerun them. The results above cover the 25 that ran. ``` Read the headline on the page together with that line. The page counts those conversations as **Lost** or ended early, never as passed or failed. If the page could not show part of a Run, the chat summary also says so, for example *The overview page could not show 2; the summary above is authoritative.* ## Next steps > [card] [Gaps and fixes](https://docs.plurai.ai/report/gaps-and-fixes.md) > > The failures grouped into gaps, worst first, with a proposed fix for each. > [card] [Run the simulation](https://docs.plurai.ai/guides/run-the-simulation.md) > > Run the suite and read the summary in chat. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.