> ## Documentation Index
> Fetch the complete documentation index at: https://cli-docs.relai.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# evaluator

> Reference for relai evaluator commands.

Use `relai evaluator` to create, upload, or remove global evaluators.

## `create`

Create a global evaluator from a prompt.

```sh theme={"system"}
relai evaluator create --prompt {PROMPT}
relai evaluator create --prompt "{describe what to evaluate}" --name {name}
relai evaluator create --prompt "{describe what to evaluate}" --agent-target {target}
```

| Option                    | Description                                                                                            |
| ------------------------- | ------------------------------------------------------------------------------------------------------ |
| `--prompt {PROMPT}`       | Evaluator behavior to generate.                                                                        |
| `--name {NAME}`           | Optional generated file name and evaluator id.                                                         |
| `--agent-target {TARGET}` | Registered target for this evaluator. Required non-interactively when multiple targets are registered. |

Creates an evaluator under `.relai/evaluators/` using the project's language. Interactive successful flows may offer to commit and upload it.

Evaluators are scoped to the selected agent target. When multiple targets are
registered, interactive runs present a single-target picker; pass
`--agent-target {target}` in non-interactive runs.

Uploaded evaluators appear on the Evaluators tab after you open your agent from the [Agents page](https://platform.relai.ai/agent).

Common failures: prompt over 8 KB, backend generation failure, evaluator validation failure, duplicate explicit name, or dirty generated evaluator paths.

## How global evaluators are applied

Evaluators created here are global: they run in every simulation of the agent
whose scope they cover, alongside the local evaluators a learning environment
or benchmark defines for itself. Because optimization evaluates through the
simulator, both kinds also score every optimizer run — see
[Evaluation during optimization](/cli/optimize#evaluation-during-optimization).

There is no per-run subset selection: all global evaluators with a matching
scope apply whenever you simulate or optimize. When an evaluator should stop
applying, [remove](#remove) it.

Scope is inferred when the evaluator is created, and evaluators run only for
simulations with the same scope. Most evaluators target end-to-end agent
behavior and run for every end-to-end simulation. An evaluator whose prompt
targets one component — for example, scoring the relevance of retrieved
documents — runs only for simulations of that component, such as
component-targeted learning environments; it does not run in end-to-end
simulations, even when the end-to-end run exercises that component. When the
intended scope is ambiguous, RELAI asks during creation; state it in the
prompt to avoid the question.

## `upload`

Upload an existing committed evaluator.

```sh theme={"system"}
relai evaluator upload --path .relai/evaluators/{name}.{extension}
relai evaluator upload --path .relai/evaluators/{name}.{extension} --agent-target {target}
```

Uploads evaluator source and git metadata to the backend.
Use `--agent-target {target}` only to upload a legacy evaluator that does not
declare its target in source; a declared target always takes precedence.

Interactive `create` runs can finish with a prompt that commits and uploads
the evaluator in one flow. Non-interactive runs — and interactive runs where
you decline that prompt — skip the upload and print the exact `git add`,
`git commit`, and `relai evaluator upload` commands to run; the evaluator is
not registered with the backend until you do. `upload` is also the recovery
path when an earlier upload failed, instead of recreating the evaluator from
scratch.

Common failures: file missing, validation failure, not committed at current `HEAD`, uncommitted changes, or backend branch conflict.

## `remove`

Remove a global evaluator by id.

```sh theme={"system"}
relai evaluator remove --id {id}
```

This is a destructive backend operation. Confirm the evaluator id before running it.