Skip to main content
RELAI turns a gap between how your agent behaves and the business requirements it should meet into a validated fix ready for review. You define the behavior you expect as learning environments, benchmarks, or global evaluators. RELAI optimizes against that definition and validates the result, then prepares any accepted improvement for review. The next gap starts the loop again, and over successive passes the agent’s behavior moves toward the requirements you care about.

​
The loop at a glance

The RELAI learning loop. A failure or gap in meeting business requirements is identified, the expected behavior is defined as learning environments, benchmarks, or global evaluators, a fix is optimized and validated, and the improvement ships as a reviewed pull request. The loop then repeats.

One pass through the loop: a requirement gap goes in, a validated and reviewed fix comes out, then the next gap starts it again.

​
Before the loop

relai init seeds, locally validates, and registers the simulator harness that lets RELAI run your git-tracked agent project. Run it once per project, before the first pass. If it pauses for input, approval, an external action, or an interruption, continue with relai init --resume.

​
The loop, step by step

Each pass moves through four stages: a requirement gap is found, the expected behavior is defined, a fix is optimized and validated, and the improvement is shipped for review. Then the loop repeats.
1

Requirement gap

Every pass starts with a real signal: a failure in production, or any gap between how the agent behaves and the business requirements it should meet. You bring the case that fell short and the outcome you expected, and the rest of the loop turns it into a fix.
2

Define expectations

Define the behavior you expect so RELAI has something concrete to target. You can express it three ways:
  • Learning environments are the unit of behavior: a repeatable scenario RELAI can simulate and optimize against. Create one from a prompt, or from a real run log plus your feedback, with relai learning-env.
  • Benchmarks are reusable evaluation suites built from a CSV, so you measure against the same cases on every pass. Register and list them with relai benchmark.
  • Evaluators score agent behavior, so the loop has a number to act on instead of a pass-or-fail judgment. Scope one to an environment or a benchmark, or register it globally so it applies across runs, with relai evaluator.
3

Optimize & validate

Measure where the agent stands, then let RELAI improve it and prove the improvement held.
  • Simulate runs your agent against the environments or benchmarks you select and reports the evaluator scores. It does not change your agent; it shows where the agent stands today. Run it with relai simulate. Simulating first is optional — optimize runs its own before-and-after evaluation.
  • Optimize runs its own before-and-after evaluation, then proposes improvements to prompts, configs, and structure and validates the result. Run it with relai optimize.
4

Review improvements

Accepted changes are committed to a separate local optimizer branch. When GitHub CLI is available and authenticated, RELAI can push that branch and open a pull request; otherwise, review and publish it yourself. Once the change merges, the next gap you spot becomes a new learning environment, and the loop runs again. Optimize’s before-and-after evaluation pass provides the attached proof, and memory carries forward, so each pass builds on what earlier ones learned.

​
Memory and tags

Memory and tags shape the whole loop rather than a single stage. Memory is markdown guidance RELAI considers during later generation and optimization, scoped to an agent, tag, component, or evaluator. Tags group learning environments into a reusable suite you can select by name when you simulate and optimize. Manage both with relai tag & memory.

Quickstart

Run init, simulate, and optimize on your own project in 5 minutes.

All commands

Every relai command with flags, examples, and failure modes.