# Eval

URL: https://docs.hanzo.ai/docs/openapi/eval

> Scoring a model on your own data, with a judge you choose.

Also for this capability: **API** · [CLI](https://docs.hanzo.ai/docs/cli/eval) · [SDKs](https://docs.hanzo.ai/docs/sdks)

Scoring a model on your own data, with a judge you choose.

> [All API references →](https://docs.hanzo.ai/docs/openapi) · [Get started →](https://docs.hanzo.ai/docs/guides)

|                |                                        |
| -------------- | -------------------------------------- |
| **Base URL**   | `https://api.hanzo.ai`                 |
| **Operations** | 16                                     |
| **Auth**       | `Authorization: Bearer $HANZO_API_KEY` |

## Specification

Specification pending — no HIP in [hanzoai/hips](https://github.com/hanzoai/hips) declares `capability: eval` yet. What this capability serves is below, from the API document; what it is — the store it owns, how it meters, what it publishes — is written as a HIP under HIP-0139.

## Four surfaces

| Surface  | Reaches this capability as                   | Coverage                                                                   |
| -------- | -------------------------------------------- | -------------------------------------------------------------------------- |
| **REST** | `eval` at its own prefix                     | 16 operations                                                              |
| **CLI**  | `hanzo eval …`                               | 16 of 16                                                                   |
| **SDK**  | `EvalApi` in every published client          | 16 methods                                                                 |
| **MCP**  | tool `eval` on `https://api.hanzo.ai/v1/mcp` | 16 operations, 0 under the document's own id — ask `describe` for the rest |

## Quickstart

```bash
export HANZO_API_KEY=sk-...   # console.hanzo.ai → API keys
```

Then the first call — a read that needs nothing but the key. `GET /v1/eval/runs`, operation `get_eval_runs`:

**CLI**

```bash
hanzo eval runs get
```

**SDK**

**.ts**

```ts
import { Configuration, EvalApi } from 'hanzoai';

const api = new EvalApi(new Configuration({ accessToken: process.env.HANZO_API_KEY }));
const { data } = await api.getEvalRuns();
```

**.py**

```python
from hanzoai.cloud import ApiClient, Configuration
from hanzoai.cloud.api import EvalApi

client = ApiClient(Configuration(access_token=os.environ["HANZO_API_KEY"]))
result = EvalApi(client).get_eval_runs()
```

**.go**

```go
cfg := hanzoai.NewConfiguration()
cfg.AddDefaultHeader("Authorization", "Bearer "+os.Getenv("HANZO_API_KEY"))
client := hanzoai.NewAPIClient(cfg)

resp, _, err := client.EvalAPI.GetEvalRuns(context.Background()).Execute()
if err != nil {
	return err
}
```

**.rs**

```rust
use hanzo_client::apis::{configuration::Configuration, eval_api};

let mut cfg = Configuration::new();
cfg.bearer_access_token = std::env::var("HANZO_API_KEY").ok();

let result = eval_api::get_eval_runs(&cfg, Default::default()).await?;
```

**.java**

```java
import ai.hanzo.cloud.ApiClient;
import ai.hanzo.cloud.api.EvalApi;

ApiClient client = new ApiClient();
client.setBearerToken(System.getenv("HANZO_API_KEY"));

var result = new EvalApi(client).getEvalRuns();
```

**HTTP**

```bash
curl https://api.hanzo.ai/v1/eval/runs \
  -H "Authorization: Bearer $HANZO_API_KEY"
```

**MCP**

MCP reaches **eval** through the `eval` tool, which names its 16 operations with its own verbs — this one among them, under a name only MCP declares. `describe` explains any of them:

```bash
curl -X POST https://api.hanzo.ai/v1/mcp \
  -H "Content-Type: application/json" \
  -d '{
       "jsonrpc": "2.0",
       "id": 1,
       "method": "tools/call",
       "params": {
         "name": "describe",
         "arguments": {
           "op": "list_eval_datasets"
         }
       }
     }'
```

Answers `200` with `object` — ok.

## Endpoints

| Endpoint                                                                                                          | What it does                                                                                                                                                                                                                                                                                               |
| ----------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [`GET /v1/eval/datasets/{name}/items`](https://docs.hanzo.ai/docs/openapi/eval/get-eval-datasets-by-name-items)   | The examples in one of your datasets — the set is named in the path, because this collection only exists inside one.                                                                                                                                                                                       |
| [`POST /v1/eval/datasets/{name}/items`](https://docs.hanzo.ai/docs/openapi/eval/post-eval-datasets-by-name-items) | Writes one graded example — its input, its expected output, free-form metadata and a status — into the dataset named in the path, and answers 201 with it.                                                                                                                                                 |
| [`GET /v1/eval/datasets/{name}`](https://docs.hanzo.ai/docs/openapi/eval/get-eval-datasets-by-name)               | Returns one dataset of the caller's org by name, together with its live item count — the one read that answers how big the set actually is.                                                                                                                                                                |
| [`DELETE /v1/eval/datasets/{name}`](https://docs.hanzo.ai/docs/openapi/eval/delete-eval-datasets-by-name)         | Removes the named dataset of the caller's org AND all of its examples, in one transaction.                                                                                                                                                                                                                 |
| [`GET /v1/eval/datasets`](https://docs.hanzo.ai/docs/openapi/eval/get-eval-datasets)                              | The datasets your org has, each with its name, description, metadata and timestamps.                                                                                                                                                                                                                       |
| [`POST /v1/eval/datasets`](https://docs.hanzo.ai/docs/openapi/eval/post-eval-datasets)                            | Writes a dataset — the named set of graded examples a run scores a model against — under the caller's org and answers 201 with it.                                                                                                                                                                         |
| [`GET /v1/eval/evaluators`](https://docs.hanzo.ai/docs/openapi/eval/get-eval-evaluators)                          | The judges your org has defined, each with its judge model, criteria and the score name it writes under.                                                                                                                                                                                                   |
| [`POST /v1/eval/evaluators`](https://docs.hanzo.ai/docs/openapi/eval/post-eval-evaluators)                        | Saves a reusable judge for the caller's org — the judge model and the written criteria it grades against — and answers 201 with it.                                                                                                                                                                        |
| [`GET /v1/eval/metrics`](https://docs.hanzo.ai/docs/openapi/eval/get-eval-metrics)                                | Your org's AI overview board over a window: totals (generations, prompt and completion tokens, cost in cents, errors, success rate, distinct models and users), a gap-filled time series, a per-model breakdown with the long tail folded into "other", and latency percentiles read from the GenAI spans. |
| [`GET /v1/eval/rubrics`](https://docs.hanzo.ai/docs/openapi/eval/get-eval-rubrics)                                | The score shapes your org has declared — each name's data type, its numeric bounds and its allowed categories.                                                                                                                                                                                             |
| [`POST /v1/eval/rubrics`](https://docs.hanzo.ai/docs/openapi/eval/post-eval-rubrics)                              | Defines the shape of one score name for the caller's org and answers 201 with it.                                                                                                                                                                                                                          |
| [`GET /v1/eval/runs`](https://docs.hanzo.ai/docs/openapi/eval/get-eval-runs)                                      | Your past runs and how they scored — the dataset and model, the judge model, how many examples were attempted and how many scored, the average score, and when it happened.                                                                                                                                |
| [`POST /v1/eval/runs`](https://docs.hanzo.ai/docs/openapi/eval/post-eval-runs)                                    | Runs a real evaluation and answers the summary when it is finished — this is synchronous work, not a job id.                                                                                                                                                                                               |
| [`GET /v1/eval/scores`](https://docs.hanzo.ai/docs/openapi/eval/get-eval-scores)                                  | The score events your org has recorded, narrowed by any of name, runName and traceId.                                                                                                                                                                                                                      |
| [`POST /v1/eval/scores`](https://docs.hanzo.ai/docs/openapi/eval/post-eval-scores)                                | Files one score event for the caller's org and answers 201 with it.                                                                                                                                                                                                                                        |
| [`GET /v1/eval/traces`](https://docs.hanzo.ai/docs/openapi/eval/get-eval-traces)                                  | The traces behind your evaluations — one per model call an evaluation made, carrying its input, output, model and timing — narrowed by any of sessionId, runName and datasetName.                                                                                                                          |

---

[All Hanzo APIs](https://docs.hanzo.ai/docs/openapi) · [Interactive reference](https://docs.hanzo.ai/docs/openapi/reference)
