Eval The `hanzo eval` commands.
Also for this capability: API · CLI · SDKs
Scoring a model on your own data, with a judge you choose.
API reference →
Every command takes --json for the raw response and --help for its own flags. Sign in once with hanzo auth login; the commands below use that session, and the org they act in is the one it carries.
Command What it does hanzo eval metricsIs your org's AI overview board over a window: totals (generations, prompt and completion tokens, cost in cents,… hanzo eval tracesIs the traces behind your evaluations — one per model call an evaluation made, carrying its input, output, model and…
Command What it does hanzo eval datasets create --name <name>Writes a dataset — the named set of graded examples a run scores a model against — under the caller's org and answers… hanzo eval datasets get <name>Returns one dataset of the caller's org by name, together with its live item count — the one read that answers how big… hanzo eval datasets items create <name>Writes one graded example — its input, its expected output, free-form metadata and a status — into the dataset named in… hanzo eval datasets items get <name>Is the examples in one of your datasets — the set is named in the path, because this collection only exists inside one. hanzo eval datasets listIs the datasets your org has, each with its name, description, metadata and timestamps. hanzo eval datasets rm <name>Removes the named dataset of the caller's org AND all of its examples, in one transaction.
Command What it does hanzo eval evaluators create --name <name>Saves a reusable judge for the caller's org — the judge model and the written criteria it grades against — and answers… hanzo eval evaluators getIs the judges your org has defined, each with its judge model, criteria and the score name it writes under.
Command What it does hanzo eval rubrics create --name <name>Defines the shape of one score name for the caller's org and answers 201 with it. hanzo eval rubrics getIs the score shapes your org has declared — each name's data type, its numeric bounds and its allowed categories.
Command What it does hanzo eval runs create --dataset <dataset> --model <model>Runs a real evaluation and answers the summary when it is finished — this is synchronous work, not a job id. hanzo eval runs getIs your past runs and how they scored — the dataset and model, the judge model, how many examples were attempted and…
Command What it does hanzo eval scores create --name <name>Files one score event for the caller's org and answers 201 with it. hanzo eval scores getIs the score events your org has recorded, narrowed by any of name, runName and traceId.
How is this guide?