Hanzo

Eval

Package eval is scoring a model on your own data, with a judge you choose.

Package eval is scoring a model on your own data, with a judge you choose.

Base URLhttps://api.hanzo.ai
Operations16
AuthAuthorization: Bearer $HANZO_API_KEY

Specification

Specification pending — no HIP in hanzoai/hips declares capability: eval yet. What this capability serves is below, from the API document; what it is — the store it owns, how it meters, what it publishes — is written as a HIP under HIP-0139.

Four surfaces

SurfaceReaches this capability asCoverage
RESTeval at its own prefix16 operations
CLIhanzo evals …16 of 16
SDKno published client declares one yet — regenerating the clients is what adds them
MCPtool evals on https://api.hanzo.ai/v1/mcp16 operations, 0 under the document's own id — ask describe for the rest

Quickstart

export HANZO_API_KEY=sk-...   # console.hanzo.ai → API keys

Then the first call — a read that needs nothing but the key. GET /v1/eval/runs, operation get_eval_runs:

hanzo evals runs get

Answers 200 with object — ok.

Endpoints

EndpointWhat it does
GET /v1/eval/datasets/{name}/itemsIs the examples in one of your datasets — the set is named in the path, because this collection only exists inside one.
POST /v1/eval/datasets/{name}/itemsWrites one graded example — its input, its expected output, free-form metadata and a status — into the dataset named in the path, and answers 201…
GET /v1/eval/datasets/{name}Returns one dataset of the caller's org by name, together with its live item count — the one read that answers how big the set actually is.
DELETE /v1/eval/datasets/{name}Removes the named dataset of the caller's org AND all of its examples, in one transaction.
GET /v1/eval/datasetsIs the datasets your org has, each with its name, description, metadata and timestamps.
POST /v1/eval/datasetsWrites a dataset — the named set of graded examples a run scores a model against — under the caller's org and answers 201 with it.
GET /v1/eval/evaluatorsIs the judges your org has defined, each with its judge model, criteria and the score name it writes under.
POST /v1/eval/evaluatorsSaves a reusable judge for the caller's org — the judge model and the written criteria it grades against — and answers 201 with it.
GET /v1/eval/metricsIs your org's AI overview board over a window: totals (generations, prompt and completion tokens, cost in cents, errors, success rate, distinct…
GET /v1/eval/rubricsIs the score shapes your org has declared — each name's data type, its numeric bounds and its allowed categories.
POST /v1/eval/rubricsDefines the shape of one score name for the caller's org and answers 201 with it.
GET /v1/eval/runsIs your past runs and how they scored — the dataset and model, the judge model, how many examples were attempted and how many scored, the average…
POST /v1/eval/runsRuns a real evaluation and answers the summary when it is finished — this is synchronous work, not a job id.
GET /v1/eval/scoresIs the score events your org has recorded, narrowed by any of name, runName and traceId.
POST /v1/eval/scoresFiles one score event for the caller's org and answers 201 with it.
GET /v1/eval/tracesIs the traces behind your evaluations — one per model call an evaluation made, carrying its input, output, model and timing — narrowed by any of…

All Hanzo APIs · Interactive reference

How is this guide?

On this page