How it works

A simulation moves through five stages: Scan → Ground → Build the suite → Run → Fix. /simulate runs them all in order. Each one is also a command of its own, and each produces what the next consumes. After every stage you get a summary in chat, and the same report page updates.

​

Scan

/simulation-scan-agent The scan is where Plurai gets to know your agent and makes it ready to simulate. It:

• Builds your agent’s topology: maps every tool, sub-agent, knowledge base and data store your agent uses, and the flows between them.
• Decides what to mock: which systems are mocked and which run live.
• Wires the digital twin: connects every mocked system to your digital twin, so your agent’s calls land there instead of in production.
• Writes the harness: the harness that drives your agent in a simulation. Your source code is never changed.

To do that, it reads your agent’s code and follows every import: the graph or agent loop, the tools, the checkpointer, the production dependencies and the knowledge bases it answers from, such as RAG over documents or a database. The harness is stored as simulate.py, next to a plurai.config.yaml that lists each tool and whether it is mocked. What you do: when you run the scan on its own, you approve the mocking map: which systems are mocked and which run for real, before any file is written. Under /simulate, the scan takes its defaults and you change anything afterwards in chat. Outcome: a harness that runs your agent with no production infrastructure and no changes to your code, and in the report, your digital twin and your knowledge base. Scan your agent →

​

Ground (optional)

/analyze-traces If you have production traces, grounding takes the simulation from realistic to hyper-realistic. It isn’t required: Plurai works well zero-shot, from your agent’s code alone. From your traces, the simulation learns not only real behavior but:

• Tool traffic: how often each tool is called and what it returns, so the mock environment behaves like your real systems.
• Anomalies: the odd responses, edge cases and noise your agent really meets.
• Speaking style, for a conversational agent: how your users actually write, so simulated users sound like them.
• Failures: what goes wrong in production, so evals target it and real failures become scenarios.

What you do: connect your observability platform (LangSmith, Langfuse, Arize or Braintrust), or point at a saved trace export. Outcome: a digital twin, scenarios and evals grounded in how your agent really behaves in production. Ground in production traces →

​

Build the suite

/simulation-generate-suite First, this stage stands up what your agent is tested against:

• The mock environment, your digital twin: the systems your agent calls, seeded with a realistic world.
• A knowledge corpus, if your agent retrieves from documents.

Then it generates the suite itself:

• Scenarios: for a conversational agent, simulated users with a goal, a persona and an opening line. For a task or workflow agent, realistic task inputs, such as a ticket, a document or a record.
• Evals: the checks that decide whether each conversation passed.

Outcome: a test suite for your agent, shown in the report under Your test suite: the agent flow graph the scenarios walk, every scenario and eval, and, if your agent answers from documents, the tested knowledge: which passages of your knowledge base the suite’s questions are answered from. Build the suite →

​

Run

/simulation-run Every scenario is played against your agent through the harness. Your agent runs on your machine, and every call it makes to a mocked system goes to your private sandbox, where each scenario starts from its own seeded world. A conversational agent talks with a simulated user: a user agent we provide that plays the scenario’s persona, pursues its goal turn by turn and can push back, change its mind or give up, like a real user. A task or workflow agent receives the scenario’s input and returns a result. Every eval scores every conversation or result.

• Three times for consistency. The final run plays each scenario three times, so the report can tell a consistent failure from a flaky one.
• In your observability platform. Record the run in LangSmith, Langfuse, Arize or Braintrust as a tracked experiment, with one record per scenario and the eval scores attached, so you can compare runs where you already work.

Outcome: pass rates and an attempt grid where every failed square opens the conversation at the turn it broke. Run the simulation →

​

Fix

/simulation-fixes The fix stage diagnoses each confirmed failure, traces it to the code that causes it, and proposes an evidence-backed fix. A gap you accept can be kept as a regression: a scenario that replays the turn where it broke, so the next suite catches it if it comes back. What you do: /simulate stops at the fix plan, so applying fixes is up to you. Review the plan and choose how involved you want to be:

• Accept the plan and the plugin applies every fix for you, then re-runs the suite to verify each one held.
• Work through it interactively: pick the fixes you want, see each change as a diff before it lands, and ask for a different approach in chat.
• Apply them yourself: use the plan’s file, line and proposed change for each issue.
• Decide later: keep the plan in the report and come back to it.

Nothing changes in your code until you accept a fix. Outcome: a ranked list of gaps with a diagnosis and a proposed fix for each, and the fixes you accepted applied and verified. Fix and keep regressions → After your first cycle, you can also replay real production failures from the turn where they broke, so the next suite tests exactly what your users hit. Replay production conversations →

​

Change anything, any time

You don’t have to approve each stage. Say what you want changed in chat: the mocking map, the mocked world, a scenario, an eval, the run or the fix plan. The flow redoes that stage and everything after it. Shape your simulation →

​

Where things run

Your agent never leaves your machine. Only the parts it would otherwise reach in production are simulated in the cloud. If you turn on anonymization, personal data is scrubbed on your machine before anything is uploaded. Without it, the sandbox still holds only simulated data: the mocked world is generated, not copied from production.

​

Where things live

See Data and privacy for exactly what leaves your machine.