Create evaluators

Saves a reusable judge for the caller's org — the judge model and the written criteria it grades against — and answers 201 with it.

POST /v1/eval/evaluators

Eval → · All API references → · Get started →

Address	`https://api.hanzo.ai/v1/eval/evaluators`
Method	`POST`
Operation	`post_eval_evaluators`
Auth	`Authorization: Bearer $HANZO_API_KEY`

Saves a reusable judge for the caller's org — the judge model and the written criteria it grades against — and answers 201 with it.

Like a dataset, the NAME is the key: re-posting a name edits that judge rather than adding a second one. Requires a validated principal; 401 without one.

Request

4 fields, body application/json (required).

Field	In	Type	Required	Description
`criteria`	body	string	—	Criteria is the written standard the judge applies; over 64 KiB is refused.
`model`	body	string	—	Model is the model that will do the grading.
`name`	body	string	yes	Name is the judge's org-unique handle, matching ^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$.
`scoreName`	body	string	—	ScoreName is the name the resulting scores are filed under.

Response

Status	Body	Meaning
`201`	eval.evaluatorView	created
`default`	problem-details	refused

201 body — 6 fields.

Field	In	Type	Always	Description
`createdAt`	body	string	—	CreatedAt is when the judge was first defined.
`criteria`	body	string	—	Criteria is the written standard the judge applies.
`model`	body	string	—	Model is the model that does the grading.
`name`	body	string	—	Name is the judge's org-unique handle.
`scoreName`	body	string	—	ScoreName is the name the resulting scores are filed under.
`updatedAt`	body	string	—	UpdatedAt is when it last changed.

Failure carries the platform error shape — see Errors.

Examples

CLI

SDK

HTTP

MCP

hanzo eval evaluators create --name <name>

.ts

.py

.go

.rs

.java

import { Configuration, EvalApi } from 'hanzoai';

const api = new EvalApi(new Configuration({ accessToken: process.env.HANZO_API_KEY }));
const { data } = await api.postEvalEvaluators({ name: "<name>" });

from hanzoai.cloud import ApiClient, Configuration
from hanzoai.cloud.api import EvalApi

client = ApiClient(Configuration(access_token=os.environ["HANZO_API_KEY"]))
result = EvalApi(client).post_eval_evaluators(name="<name>")

cfg := hanzoai.NewConfiguration()
cfg.AddDefaultHeader("Authorization", "Bearer "+os.Getenv("HANZO_API_KEY"))
client := hanzoai.NewAPIClient(cfg)

resp, _, err := client.EvalAPI.PostEvalEvaluators(context.Background()).Execute()
if err != nil {
	return err
}

use hanzo_client::apis::{configuration::Configuration, eval_api};

let mut cfg = Configuration::new();
cfg.bearer_access_token = std::env::var("HANZO_API_KEY").ok();

let result = eval_api::post_eval_evaluators(&cfg, Default::default()).await?;

import ai.hanzo.cloud.ApiClient;
import ai.hanzo.cloud.api.EvalApi;

ApiClient client = new ApiClient();
client.setBearerToken(System.getenv("HANZO_API_KEY"));

var result = new EvalApi(client).postEvalEvaluators();

curl -X POST https://api.hanzo.ai/v1/eval/evaluators \
  -H "Authorization: Bearer $HANZO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
       "name": "<name>"
     }'

MCP reaches eval through the eval tool, which names its 16 operations with its own verbs — this one among them, under a name only MCP declares. describe explains any of them:

curl -X POST https://api.hanzo.ai/v1/mcp \
  -H "Content-Type: application/json" \
  -d '{
       "jsonrpc": "2.0",
       "id": 1,
       "method": "tools/call",
       "params": {
         "name": "describe",
         "arguments": {
           "op": "list_eval_datasets"
         }
       }
     }'

Eval API · All Hanzo APIs · Interactive reference

Was this page useful?

PreviousCreate datasets NextCreate rubrics