Eval
The hanzo eval commands.
Also for this capability: API · CLI · SDKs
Scoring a model on your own data, with a judge you choose.
hanzo eval datasets listEvery command takes --json for the raw response and --help for its own flags. Sign in once with hanzo auth login; the commands below use that session, and the org they act in is the one it carries.
| Command | What it does |
|---|---|
hanzo eval metrics | Your org's AI overview board over a window: totals (generations, prompt and completion tokens, cost in cents, errors, success rate, distinct models and users), a gap-filled time series, a per-model breakdown with the long tail folded into "other", and latency percentiles read from the GenAI spans. |
hanzo eval traces | The traces behind your evaluations — one per model call an evaluation made, carrying its input, output, model and timing — narrowed by any of sessionId, runName and datasetName. |
Datasets
| Command | What it does |
|---|---|
hanzo eval datasets create --name <name> | Writes a dataset — the named set of graded examples a run scores a model against — under the caller's org and answers 201 with it. |
hanzo eval datasets get <name> | Returns one dataset of the caller's org by name, together with its live item count — the one read that answers how big the set actually is. |
hanzo eval datasets items create <name> | Writes one graded example — its input, its expected output, free-form metadata and a status — into the dataset named in the path, and answers 201 with it. |
hanzo eval datasets items get <name> | The examples in one of your datasets — the set is named in the path, because this collection only exists inside one. |
hanzo eval datasets list | The datasets your org has, each with its name, description, metadata and timestamps. |
hanzo eval datasets rm <name> | Removes the named dataset of the caller's org AND all of its examples, in one transaction. |
Evaluators
| Command | What it does |
|---|---|
hanzo eval evaluators create --name <name> | Saves a reusable judge for the caller's org — the judge model and the written criteria it grades against — and answers 201 with it. |
hanzo eval evaluators get | The judges your org has defined, each with its judge model, criteria and the score name it writes under. |
Rubrics
| Command | What it does |
|---|---|
hanzo eval rubrics create --name <name> | Defines the shape of one score name for the caller's org and answers 201 with it. |
hanzo eval rubrics get | The score shapes your org has declared — each name's data type, its numeric bounds and its allowed categories. |
Runs
| Command | What it does |
|---|---|
hanzo eval runs create --dataset <dataset> --model <model> | Runs a real evaluation and answers the summary when it is finished — this is synchronous work, not a job id. |
hanzo eval runs get | Your past runs and how they scored — the dataset and model, the judge model, how many examples were attempted and how many scored, the average score, and when it happened. |
Scores
| Command | What it does |
|---|---|
hanzo eval scores create --name <name> | Files one score event for the caller's org and answers 201 with it. |
hanzo eval scores get | The score events your org has recorded, narrowed by any of name, runName and traceId. |
Was this page useful?