Skip to main content
Use relai benchmark to register and list benchmark suites.

​
register

Register a benchmark from CSV data. 
relai benchmark register --csv {PATH}
Benchmark registration can take a while when RELAI needs to generate benchmark support from larger CSV data or evaluator context.
OptionDescription
--csv {PATH}Benchmark CSV file.
--name {NAME}Optional benchmark name, id, and output filename.
--evaluator-ref {PATH}Optional Python evaluator reference or context file. May be repeated.
--prompt {TEXT}Optional inline generation context.
relai benchmark register --csv {path/to/data.csv} --name {name}
relai benchmark register \
  --csv {path/to/data.csv} \
  --evaluator-ref {path/to/evaluator.py} \
  --prompt "{scoring guidance}"
Creates benchmark files under .relai/benchmarks/ and may upload benchmark artifacts depending on the flow. Uploaded benchmarks appear on the Benchmarks tab after you open your agent from the Agents page. Common failures: invalid CSV, missing required columns, generation ambiguity, validation failure, dirty generated paths, or backend upload failure.

​
list

List registered benchmarks. 
relai benchmark list
Read-only. Common failures are malformed benchmark files or missing simulator dependencies.