Observability platforms

Plurai Simulation works natively with the observability platform your team already uses: LangSmith, Langfuse, Arize and Braintrust. There is no proxy to deploy and no code to add to your agent. The plugin and the SDK talk to the platform directly from your machine, in three ways:

Analyze production traces to ground the mock environment, scenarios and evals in how your agent really behaves and how your users really talk. This is the optional Ground stage (/analyze-traces). See Ground in production traces.
Replay real conversations from where they broke. A production conversation becomes a scenario that resumes at the failing turn, so the failure reproduces and gets a fix. See Replay production conversations.
Record Runs as experiments, one record per scenario with the eval scores as feedback, so simulation results sit next to your production traces. See Record Runs.

Your platform credentials stay on your machine in every direction. The SDK reads the traces locally and never writes the keys to disk or uploads them.

Platform	Analyze traces	Replay conversations	Record Runs
LangSmith	Yes	Yes, thread exports from LangGraph agents	Yes
Langfuse	Yes	Not yet	Yes
Arize	Coming soon	Yes, OpenInference span exports	Yes
Braintrust	Coming soon	Through an adapter	Yes
Saved trace export	Yes	Yes	—
Any other format	—	Through an adapter	—

“Through an adapter” means you run /author-trace-adapter once to teach the converter the export’s format. See Teach the converter a new format.

From traces to fixes

Every path ends in the same place: a confirmed gap, traced to your code, with a fix.

Connect

Point the plugin at your platform and project. Credentials come from your environment or the plugin asks.

Analyze

/analyze-traces exports the traces locally and writes a trace brief: your agent’s capabilities, anomalies, how your users talk, and the failures worth testing. The suite is built grounded in it.

Replay

The failures in the brief become scenarios that resume at the turn where each one broke.

Run and fix

/simulation-run plays them with the rest of the suite. /simulation-fixes traces each failure to the code and proposes a fix, and an accepted fix is kept as a regression.

Record

Each Run can be recorded back to your platform as an experiment, so you can compare it with production.

Analyze production traces

Run the Ground stage and say where your traces are:

/analyze-traces

Or in chat: “Ground the simulation in our LangSmith traces from the billing-prod project, only failed runs.” The plugin builds a source config for the SDK, runs plurai ingest-traces locally, and uploads the exported traces under your workspace’s anonymization posture. You approve the command before it runs. See Ground in production traces.

LangSmith
Langfuse
Saved trace export

Install the extra in the SDK’s virtual environment: plurai[langsmith].Credentials. The plugin reads LANGSMITH_API_KEY from your environment when it is set, or asks you.Source config (--source-kind langsmith):

Key	Required	Meaning
`api_key`	Yes	Your LangSmith API key. Stays local.
`project`	Yes	The project (session) to export from.
`api_url`	No	A self-hosted LangSmith endpoint.
`filter`	No	A LangSmith run-filter string that narrows the export, for example `eq(status, "error")`.
`thread_ids`	No	Export these conversations whole, one file per thread, instead of the most recent traces.

{"api_key": "<key>", "project": "billing-prod", "filter": "eq(status, \"error\")"}

Install the extra in the SDK’s virtual environment: plurai[langfuse]. The SDK requires Langfuse 4.x.Credentials. The plugin reads LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY and LANGFUSE_HOST from your environment when they are set, or asks you.Source config (--source-kind langfuse):

Key	Required	Meaning
`public_key`	Yes	The project’s public key. The key pair selects the project.
`secret_key`	Yes	The project’s secret key. Stays local.
`host`	No	A self-hosted Langfuse URL. Without it, the Langfuse client reads `LANGFUSE_HOST`.
`environment`	No	Export only traces stamped with this environment.
`name`	No	Export only traces with this name.

Without environment or name, the export takes the project’s most recent traces. Langfuse Cloud is read through its Observations API v2; a self-hosted Langfuse v3 server is read through the v1 trace API.

{"public_key": "<pk>", "secret_key": "<sk>", "environment": "production"}

Re-use a trace export you already saved, with no live connection to the platform. The directory must be one an earlier plurai ingest-traces run wrote: a traces/ folder of *.json files and a manifest.json.Source config (--source-kind export_dir):

Key	Required	Meaning
`path`	Yes	The export directory.
`thread_ids`	No	Read only these conversations.

No other key is accepted: a saved export cannot be re-queried, so a filter would select nothing. Write the new export to a different --out-dir than the one you read from.

plurai ingest-traces --source-kind export_dir \
  --source-config export.json --out-dir plurai_sim/traces

The uploaded part is byte-identical to a live read of the same traces. Requires SDK 0.28.4 or newer; plurai capabilities lists export-dir-traces.

An export that finds zero traces stops with an error naming the config keys it used, so a wrong project, filter or key is caught before anything is uploaded. For other trace formats (a session log, an in-house exporter’s JSON), /author-trace-adapter teaches the converter your format.

Replay conversations

/trace-to-scenarios turns a conversation into a scenario that resumes where it broke. It reads:

LangSmith: thread exports from LangGraph agents, including the files /analyze-traces exported. This is the default format.
Arize: OpenInference span trees, the spans an OpenInference-instrumented agent emits. Export them from Arize or Phoenix as JSON, either as a bare list of spans or as a {trace_id, spans} object, and pass --source-format openinference.
Anything else: run /author-trace-adapter first, then pass the adapter’s name with --source-format.

Langfuse traces can’t be replayed yet. See Replay production conversations for the whole flow.

Record Runs

Recording is opt-in. A Run records nothing unless you declare it, even when platform keys are set:

Where	How
Flag	`plurai-run-simulation --observability` (or `--no-observability`)
Environment	`PLURAI_OBSERVABILITY=recording`, exported in the real environment. A `.env` file does not count.
Workspace	Write `recording` into `.observability` beside `simulate.py`.
Default	Nothing is recorded.

/simulation-run asks whether to record the Run as an experiment. Then pick one platform: exactly one <SINK>__ENABLED flag may be set, and enabling two fails at startup.

LangSmith
Langfuse
Arize
Braintrust

Install plurai[langsmith], then set:

export LANGSMITH__ENABLED=true
export LANGSMITH__API_KEY=<key>
export LANGSMITH__PROJECT=<project>    # optional
export LANGSMITH__DATASET=<dataset>    # optional

The scenario set is uploaded as a dataset before the Run starts.

Install plurai[langfuse], then set:

export LANGFUSE__ENABLED=true
export LANGFUSE__PUBLIC_KEY=<pk>
export LANGFUSE__SECRET_KEY=<sk>
export LANGFUSE__HOST=<url>            # optional
export LANGFUSE__ENVIRONMENT=<env>     # optional
export LANGFUSE__PROJECT=<project>     # optional
export LANGFUSE__DATASET=<dataset>     # optional

Install plurai[arize], then set:

export ARIZE__ENABLED=true
export ARIZE__API_KEY=<key>
export ARIZE__SPACE_ID=<space id>
export ARIZE__PROJECT_NAME=<project>   # optional
export ARIZE__ENDPOINT=<url>           # optional

Arize runs the scenario set as one experiment, so a Run recorded to Arize takes one trial per scenario.

Install plurai[braintrust], then set:

export BRAINTRUST__ENABLED=true
export BRAINTRUST__API_KEY=<key>
export BRAINTRUST__PROJECT=<project>   # optional

Reading traces and recording Runs use different variable names. Trace export reads the platform’s usual one-underscore names (LANGSMITH_API_KEY, LANGFUSE_PUBLIC_KEY). Recording a Run reads only the two-underscore names above (LANGSMITH__API_KEY, LANGFUSE__PUBLIC_KEY).

Where you leave a two-underscore credential unset, the vendor’s own client may still read its usual one-underscore variable. A missing package or a platform error never breaks a Run: the recording is best-effort. After a Run, when you correct an eval’s verdict, plurai correct-scores overwrites that eval’s score on Langfuse or LangSmith. See SDK CLI.

Next steps

Ground in production traces

The Ground stage, step by step.

Replay production conversations

Resume a real conversation at the turn where it broke.

Data and privacy

What leaves your machine, and the anonymization posture.

Overview

Coding agents

Agent frameworks

Trace sources

From traces to fixes

Analyze production traces

Replay conversations

Record Runs

Next steps

Ground in production traces

Replay production conversations

Data and privacy

​From traces to fixes

​Analyze production traces

​Replay conversations

​Record Runs

​Next steps

Ground in production traces

Replay production conversations

Data and privacy

From traces to fixes

Analyze production traces

Replay conversations

Record Runs

Next steps