Saves a reusable judge for the caller's org — the judge model and the written…
Saves a reusable judge for the caller's org — the judge model and the written criteria it grades against — and answers 201 with it.
POST /v1/eval/evaluators
| Address | https://api.hanzo.ai/v1/eval/evaluators |
| Method | POST |
| Operation | post_eval_evaluators |
| Auth | Authorization: Bearer $HANZO_API_KEY |
Saves a reusable judge for the caller's org — the judge model and the written criteria it grades against — and answers 201 with it.
Like a dataset, the NAME is the key: re-posting a name edits that judge rather than adding a second one. Requires a validated principal; 403 without one.
Request
4 fields, body application/json (required).
| Field | In | Type | Required | Description |
|---|---|---|---|---|
criteria | body | string | — | Criteria is the written standard the judge applies; over 64 KiB is refused. |
model | body | string | — | Model is the model that will do the grading. |
name | body | string | yes | Name is the judge's org-unique handle, matching ^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$. |
scoreName | body | string | — | ScoreName is the name the resulting scores are filed under. |
Response
| Status | Body | Meaning |
|---|---|---|
201 | evaluatorView | created |
201 body — 6 fields.
| Field | In | Type | Always | Description |
|---|---|---|---|---|
createdAt | body | string | — | CreatedAt is when the judge was first defined. |
criteria | body | string | — | Criteria is the written standard the judge applies. |
model | body | string | — | Model is the model that does the grading. |
name | body | string | — | Name is the judge's org-unique handle. |
scoreName | body | string | — | ScoreName is the name the resulting scores are filed under. |
updatedAt | body | string | — | UpdatedAt is when it last changed. |
Failure carries the platform error shape — see Errors.
Examples
hanzo evals evaluators create --name <name>import { Configuration, EvalApi } from 'hanzoai';
const api = new EvalApi(new Configuration({ accessToken: process.env.HANZO_API_KEY }));
const { data } = await api.postEvalEvaluators({ name: "<name>" });from hanzoai.cloud import ApiClient, Configuration
from hanzoai.cloud.api import EvalApi
client = ApiClient(Configuration(access_token=os.environ["HANZO_API_KEY"]))
result = EvalApi(client).post_eval_evaluators(name="<name>")cfg := cloud.NewConfiguration()
cfg.AddDefaultHeader("Authorization", "Bearer "+os.Getenv("HANZO_API_KEY"))
client := cloud.NewAPIClient(cfg)
resp, _, err := client.EvalAPI.PostEvalEvaluators(context.Background()).Execute()
if err != nil {
return err
}use hanzo_cloud::apis::{configuration::Configuration, eval_api};
let mut cfg = Configuration::new();
cfg.bearer_access_token = std::env::var("HANZO_API_KEY").ok();
let result = eval_api::post_eval_evaluators(&cfg, Default::default()).await?;import ai.hanzo.cloud.ApiClient;
import ai.hanzo.cloud.api.EvalApi;
ApiClient client = new ApiClient();
client.setRequestInterceptor(b -> b.header("Authorization", "Bearer " + System.getenv("HANZO_API_KEY")));
var result = new EvalApi(client).postEvalEvaluators();The method above is the one at the current release of the document. [email protected] (npm) and [email protected] (PyPI) were generated from an earlier release, where this operation carried a different id, so it spells the method differently — regenerating the clients is what makes the two agree. SDKs →
curl -X POST https://api.hanzo.ai/v1/eval/evaluators \
-H "Authorization: Bearer $HANZO_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"name": "<name>"
}'The door reaches eval through the evals tool, which names its 16 operations with its own verbs — this one among them, under a name only the door declares. describe explains any of them:
curl -X POST https://api.hanzo.ai/v1/mcp \
-H "Content-Type: application/json" \
-d '{
"jsonrpc": "2.0",
"id": 1,
"method": "tools/call",
"params": {
"name": "describe",
"arguments": {
"op": "list_eval_datasets"
}
}
}'How is this guide?