Skip to main content
relai simulate runs selected learning environments against the generated simulator.

​
When to simulate

Simulation measures; it never changes your agent. Run it to see where the agent stands against selected environments or benchmarks, or to re-check behavior after changes merge. It is not a prerequisite for optimization. relai optimize runs its own before/after evaluation passes and does not reuse earlier simulation results, so when you already know you want the optimizer, you can skip straight to it.

​
Select environments

At least one selector should identify what to run.

​
Runtime and output

​
Environment variables

Set the runtime variables your simulator needs before running a simulation or optimization. Common examples include a model-provider key such as OPENAI_API_KEY, ANTHROPIC_API_KEY, or GOOGLE_API_KEY, and credentials for tools your agent calls. Use the variable names listed in .relai/simulator.env.example; the examples below illustrate common names. Choose one of the first two options for most workflows: use a local file when the values should persist across development runs, or set them in the command environment for a one-off run or CI. Use --env-file or --env only when you need a different file or an invocation-specific override.

​
Persistent local values: .relai/simulator.env

relai init creates .relai/simulator.env.example with the variables your simulator expects. Copy it to .relai/simulator.env, then replace the placeholders with local values. RELAI loads this file automatically for both relai simulate and relai optimize when it exists.
Use this file when the same values are needed across local runs. It accepts one KEY=VALUE assignment per line; blank lines, comments, and an optional export prefix are allowed.

​
One-off or CI values: the command environment

For temporary values, set a variable inline with the command or export it in the shell before running RELAI. This works for both simulation and optimization and is usually the better choice for CI, where the platform injects secrets as environment variables.

​
Alternate file or command-specific override

Pass --env-file when the values live outside the default local file, or use --env KEY=VALUE for a value that applies only to that command.
RELAI starts with variables from the process environment, then loads .relai/simulator.env, then each --env-file, and finally each --env value. When the same key appears more than once, the later source takes precedence. Keep secrets out of version control. .relai/simulator.env is local-only and should not be committed; commit .relai/simulator.env.example only with placeholder values. In CI, provide secrets through the platform’s secure environment-variable or secret mechanism rather than a tracked file.

​
Examples

​
What to expect

  • terminal simulation summary.
  • optional result JSON when --result-json is provided.
  • local run artifacts under .relai/runs/ when produced by the simulator.
  • backend publication of the core result by default. Missing, unreadable, or oversized optional report and transcript attachments produce warnings but do not fail a publication that accepts the core result.
A simulation is locally successful only after the simulator exits successfully and RELAI parses its result JSON. --no-backend-upload makes that local result the complete successful outcome. With publication enabled, configuration, target lookup, result preparation, or publication failures exit with status 1 after the local summary. RELAI retains the raw result and available transcript under .relai/runs/ and prints their paths. A POST transport or retryable-server failure may have an unknown outcome; RELAI does not replay that write because it could create a duplicate. Batch runs continue after individual execution or publication failures. Their suite report keeps every valid local result: success_count counts parsed local results, failure_count counts local execution failures, and publication_failure_count separately counts backend publication failures. The command exits with status 1 if either kind of failure occurred. Uploaded simulation results appear on the Simulations tab after you open your agent from the Agents page. Common failures: missing simulator, unknown tag or benchmark, missing runtime credentials, invalid environment file, missing or invalid simulator result JSON, simulator dependency failure, or backend publication failure. relai simulate does not intentionally edit agent source. If simulator execution fails, RELAI may ask the configured backend to repair selected learning environment or simulator code and retry.