Skip to main content
Use relai evaluator to create, upload, or remove global evaluators.

​
create

Create a global evaluator from a prompt. 
relai evaluator create --prompt {PROMPT}
relai evaluator create --prompt "{describe what to evaluate}" --name {name}
OptionDescription
--prompt {PROMPT}Evaluator behavior to generate.
--name {NAME}Optional generated file name and evaluator id.
Creates a Python evaluator under .relai/evaluators/. Interactive successful flows may offer to commit and upload it. Uploaded evaluators appear on the Evaluators tab after you open your agent from the Agents page. Common failures: prompt over 8 KB, backend generation failure, evaluator validation failure, duplicate explicit name, or dirty generated evaluator paths.

​
upload

Upload an existing committed evaluator. 
relai evaluator upload --path .relai/evaluators/{name}.py
Uploads evaluator source and git metadata to the backend. Common failures: file missing, validation failure, not committed at current HEAD, uncommitted changes, or backend branch conflict.

​
remove

Remove a global evaluator by id. 
relai evaluator remove --id {id}
This is a destructive backend operation. Confirm the evaluator id before running it.