> ## Documentation Index
> Fetch the complete documentation index at: https://cli-docs.relai.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Changelog

> New features and bug fixes in RELAI CLI releases.

Follow the latest improvements to RELAI. Each release lists new features and bug fixes.

To install the latest version, run:

```bash theme={"system"}
relai update
```

Release notes appear here after each CLI release is published.

**v0.2.16**

2026-10-30

## RELAI CLI 0.2.16

### What's new

* Added relai custom-adapter validate run and relai custom-adapter validate results. They check that a custom adapter produces valid, useful results before you optimize, and they explain why any task failed or went unscored. A read-only model review adds advisory findings, and --offline skips it.
* relai optimize --full is now generally available and no longer needs RELAI\_EXPERIMENTAL. Full runs now include reviewed report summaries and test quality findings, even when no change is accepted.

### Bug fixes

* Custom adapter failures now name the adapter, the stage, the task and the result path, which makes problems easier to track down.

**v0.2.15**

2026-10-29

## RELAI CLI 0.2.15

### Bug fixes

* \`relai init\` now sets up a starter test template when your smoke test includes code-based checks. Previously it stopped with "No test template was seeded".
* \`relai init\` no longer rejects a successful smoke trial when the agent's tool call includes extra fields inside a nested argument. Declared values must still match, so the init workflow no longer pauses on a valid trial.

**v0.2.14**

2026-10-28

## RELAI CLI 0.2.14

### What's new

* Installing the RELAI plugin for Claude Code now registers it with the claude command when it is available and confirms it is installed and enabled. A plugin you disabled stays disabled, and the status message tells you how to turn it on.
* When the RELAI CLI is missing, the plugin skills now show you the install command to run in your own terminal instead of running the installer for you.
* The RELAI plugin packages now include an icon.

### Bug fixes

* When the time limit or Ctrl-C interrupted a run partway, it now pauses cleanly so that --resume can continue it. Before, some runs failed with a search error or saved a risk with an unknown revision.
* Resuming after a crash or interruption now finishes the interrupted work in discovery, ingest, wiki bootstrap, init and benchmark register and update. Before, some of these runs failed on resume or left a stale wiki page or an orphaned stage behind.
* If the time limit expired while init was seeding the test template, init used to finish without the template and never retry. It now stops so that relai init --resume can seed it.
* The evaluator create token-usage line now goes to stderr instead of stdout, so it no longer comes before the completed document.
* Discovery and wiki reconcile now report the right workflow name and a completion document when they run with --agent-interaction, including after a resume.
* A time limit hit during a queued wiki update in suggestion accept now points to the correct resume command.

**v0.2.13**

2026-10-27

## RELAI CLI 0.2.13

### What's new

* Simulation result files are much smaller. Large agent tests no longer produce files of 150 MB or more. Older results can still be retried, but older CLI versions will ask you to upgrade to read new results.
* Agent-test templates are now generally available. You can list, show, create and update templates, use them when creating tests or accepting suggestions, and pick a template version when updating a test. No experimental flag is needed.
* relai init seeds a basic template test after its smoke test when permissions allow it.
* relai init pauses early when a required value, such as an API key, is missing. Add it to .relai/simulator.env and resume. init also no longer asks which target agent to describe, and it uses the targets in your config.
* Trials started from init use a tool-permission policy closer to your real setup. Only actions that reach outside the disposable trial are refused.
* Agent tests built from review comments or other feedback no longer reveal the reviewer's diagnosis or suggested fix to the agent. The feedback can still shape the scoring.
* Harbor trials now report cost and token totals when the agent's trajectory records usage. Adapters you already generated need to be re-authored by init to report them.
* Test discovery is stricter about confirming a risk. One triggered trial gives medium confidence, and high confidence needs two.
* relai guide is now a public command. Coding agents use it to keep working when the installed RELAI plugin is out of date.
* Coding agents now measure related existing tests before a planned behavior change and rerun them afterward. Run relai policy install to pick this up on existing installs.
* Harbor trials, workspace setup and runtime checks now work better with Docker Desktop and with remote or VM-backed Docker.
* relai optimize reinstalls a Harbor project's dependencies only when its dependency files or install and check scripts have changed.

### Bug fixes

* Fixed a false "workspace mount" warning on Docker Desktop.
* Generated adapters no longer merge several model replies into one after tool calls.
* A generated adapter file missing from the test package is now caught during packaging, with the paths shown, instead of failing when the trial launches.
* Retrying a benchmark sample whose task template declares a trajectory example no longer fails.
* A resumed relai init that validates successfully now counts as a successful init.
* If your .gitignore already ignores the whole .relai/ folder, RELAI no longer adds its own ignore entries. A fresh init notes that this rule also hides .relai/evals-wiki/.
* Retrying test create, test update, benchmark register or benchmark update after a crash during the first save now removes the leftover draft.

**v0.2.12**

2026-10-27

## RELAI CLI 0.2.12

### What's new

* Messages, progress output and errors now say "agent test" instead of "learning environment". Existing projects and paused sessions keep working.
* Workflow agents (init, benchmark, agent-test) can now ask several questions in one pause. You answer them together and the run resumes once.
* relai setup --check now verifies that tools actually work: Harbor starts, Python 3.12 is available, and Docker is running Linux containers. Each failure comes with evidence and a fix for your operating system.
* relai init runs the same checks before any model work and lists every problem if your machine can't run a trial.
* Agent-test planning now lists what a test needs beyond the default environment, such as extra services, mounts, or Docker or network access. It compares those needs with what is supported before you approve, and offers revision options if something isn't.
* Test and evaluator scoring now gives a criterion its minimum score when the agent delivers nothing to grade. Rules that only prohibit something are still met when the agent does nothing.
* Creating a benchmark or agent test with empty tags is now rejected.
* relai optimize --fast now picks only regression checks that can actually be scored. If none qualify, it reports "focus verified; no independent regression coverage" instead of running pointless simulations.

### Bug fixes

* Full-mode optimization can now see and edit coding-agent skills, agent definitions and instruction files, such as those under .agents, .claude, .codex, .cursor and .github.
* Optimization now keeps your own benchmark and test tags instead of replacing them with generated ones.
* EvalsWiki bootstrap now produces requirements when your project instructions name a coding agent as the target.
* A new project is no longer forced into the legacy layout just because .relai/inputs/ contains your own files.
* Test, evaluator and benchmark names are no longer silently cut to 48 characters. Names that are too long are rejected up front.
* Managed Harbor builds no longer fail on large file trees, and files such as .gitignore, .github files and .env.example are no longer rejected.
* Harbor now always runs on Python 3.12, so relai init no longer fails when uv picks an older default Python.
* During relai init and agent-test authoring, the agent can read its own validation trial results, so it no longer asks you to export run files.
* When a test's own setup is broken, preflight now fails early with the failing command, its exit status, and the start and end of its output.
* A preflight check that leaves a background process running no longer hangs.
* relai optimize --fast now keeps the earlier regression-check result when a later attempt couldn't recheck it, and the completion summary shows both.
* In sessions without a native question tool, bundled skills now continue with an answer you already gave instead of stopping.
* Bundled Codex guidance now waits for running RELAI commands in long blocks instead of many short polls.

**v0.2.11**

2026-10-26

## RELAI CLI 0.2.11

### What's new

* Windows: relai completions now supports PowerShell, and relai update and relai uninstall work on Windows.
* Added relai regression select, which suggests which existing tests and benchmark rows to re-run after a code change.
* relai optimize now has --fast and --full modes and asks you to choose if you don't. Full mode stops early by default, and --no-early-stop turns that off.
* relai simulate can now run selected custom-adapter tasks and individual benchmark samples.
* Optimization results now compare only runs that produced a real verdict. Runs with no verdict are reported as unscored instead of counted as zero.
* relai setup --agent can now sign in from a coding agent's shell. Under --agent-interaction, commands that need a sign-in pause up front and point you to relai setup.
* relai init now checks that Docker is running before it starts. It also names missing simulator values, such as API keys, as soon as they are known.
* Test creation, discovery, simulation, and optimization are faster. Docker images are reused across trials and commands, AI judges score at the same time, and several rebuilds and validations are skipped when nothing changed. A new keep\_images setting under \[harbor] in .relai/config.toml controls whether built images are kept.
* Limited benchmark runs now prepare only the selected rows.
* Benchmark registration and updates now reject invalid or conflicting evaluator references before publishing.
* Evaluator create and update summaries now report how many tests and benchmark samples the evaluator applies to.
* relai wiki list now shows which rows are displayed and the next --offset. New options: relai wiki update --discard and relai wiki reconcile --restart. relai wiki bootstrap --discard now confirms and exits 0.

### Bug fixes

* Failing trials now report adapter failures right away, with the exit status and error output, instead of waiting up to 180 seconds. Failed trials also show the cause and the verifier's output more reliably.
* relai optimize now stops with a clear message when Docker or the worker image is unavailable. It rebuilds custom adapters after every code change, rejects oversized custom instructions up front, and exits non-zero when every attempt fails. Narrower runs no longer erase learning for tests they didn't select.
* relai simulate now fails when any selected run produces no result. It checks required runtime values before starting and exits 130 on Ctrl-C. --retry-from-outcome refuses selectors that differ from the original run.
* Ctrl-C and timeouts are now reported as cancellation (exit 130) or timeout (exit 124) across optimize, simulate, Wiki, setup, tool checks, and workflow commands. Progress is kept so you can use --resume. When there is nothing to resume, you are asked to re-run instead.
* Resume instructions in pause messages now name the correct command, and a paused runtime repair in simulate points you to relai init --resume.
* Damaged benchmark or test packages no longer hide everything else in list commands. They are named, and the command exits non-zero.
* relai update now works in projects written by a newer RELAI version. relai uninstall cleans every known shell profile and leftover temporary files, and exits non-zero if a step fails.
* relai evaluator remove now refuses to delete an evaluator that a benchmark uses and can finish an interrupted removal. relai benchmark remove can also be resumed.
* Duplicate CSV column names in benchmark datasets are now rejected. Special files such as FIFOs are rejected when benchmarks, evaluators, and tests are copied.
* Background runs no longer expose clarification answers, and their files are readable only by their owner. On Windows, session and credential files are private again, and timeouts now stop child processes.
* A saved machine API key now works for optimizer calls. Credential handling is now consistent across auth, update, setup, and optimizer sign-in.
* Model failures such as spending-limit rejections no longer trigger unrelated repair attempts. Init failures now name missing dependencies and show the step that failed.
* Discovery no longer fails README validation after the builder rewords the approved description. A failed discovery candidate no longer retires a risk that a published test already covers.
* A closed stdout (for example relai ... | head) no longer hides the real error.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.