Admits and queues a benchmark run against a model or your own endpoint, and…
Admits and queues a benchmark run against a model or your own endpoint, and answers 202 with the receipt.
POST /v1/benchmark/runs
| Address | https://api.hanzo.ai/v1/benchmark/runs |
| Method | POST |
| Operation | post_benchmark_runs |
| Auth | Authorization: Bearer $HANZO_API_KEY |
Admits and queues a benchmark run against a model or your own endpoint, and answers 202 with the receipt.
It is an ADMISSION, not a result: the work is done by the harness afterwards and the numbers appear on the leaderboard as it completes them.
Cost is bounded by the store rather than by a quota: attempts are append-only and keyed by (benchmark, item, model), so an (item, model) pair already attempted is skipped instead of re-spent, and re-queuing the same run is close to free.
Validation is up front and total — a request with neither model nor endpoint is a 400, one with no benchmarks is a 400, and any benchmark id outside the catalog is a 422 naming exactly which ids were unknown, so a typo never silently queues a partial run.
Request
4 fields, body application/json (required).
| Field | In | Type | Required | Description |
|---|---|---|---|---|
attempts | body | integer | — | Attempts is how many times to try each item; the harness's default applies when it is omitted. |
benchmarks | body | string[] | yes | Benchmarks are the catalog ids to run. |
endpoint | body | string | — | Endpoint is your own chat-completions URL, for benchmarking a model this arena does not host. |
model | body | string | — | Model is the catalog model id to run. |
Response
| Status | Body | Meaning |
|---|---|---|
202 | admission | accepted |
202 body — 5 fields.
| Field | In | Type | Always | Description |
|---|---|---|---|---|
benchmarks | body | string[] | — | Benchmarks are the catalog ids admitted. |
endpoint | body | string | — | Endpoint is the caller's own endpoint the run targets. |
model | body | string | — | Model is the catalog model the run targets. |
note | body | string | — | Note explains what admission does and does not promise. |
status | body | string | — | Status is "queued": the run is admitted, not finished. |
Failure carries the platform error shape — see Errors.
Examples
hanzo benchmark runs --benchmarks '["<benchmarks>"]'import { Configuration, BenchmarkApi } from 'hanzoai';
const api = new BenchmarkApi(new Configuration({ accessToken: process.env.HANZO_API_KEY }));
const { data } = await api.postBenchmarkRuns({ benchmarks: ["<benchmarks>"] });from hanzoai.cloud import ApiClient, Configuration
from hanzoai.cloud.api import BenchmarkApi
client = ApiClient(Configuration(access_token=os.environ["HANZO_API_KEY"]))
result = BenchmarkApi(client).post_benchmark_runs(benchmarks=["<benchmarks>"])cfg := cloud.NewConfiguration()
cfg.AddDefaultHeader("Authorization", "Bearer "+os.Getenv("HANZO_API_KEY"))
client := cloud.NewAPIClient(cfg)
resp, _, err := client.BenchmarkAPI.PostBenchmarkRuns(context.Background()).Execute()
if err != nil {
return err
}use hanzo_cloud::apis::{configuration::Configuration, benchmark_api};
let mut cfg = Configuration::new();
cfg.bearer_access_token = std::env::var("HANZO_API_KEY").ok();
let result = benchmark_api::post_benchmark_runs(&cfg, Default::default()).await?;import ai.hanzo.cloud.ApiClient;
import ai.hanzo.cloud.api.BenchmarkApi;
ApiClient client = new ApiClient();
client.setRequestInterceptor(b -> b.header("Authorization", "Bearer " + System.getenv("HANZO_API_KEY")));
var result = new BenchmarkApi(client).postBenchmarkRuns();curl -X POST https://api.hanzo.ai/v1/benchmark/runs \
-H "Authorization: Bearer $HANZO_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"benchmarks": [
"<benchmarks>"
]
}'The door reaches benchmark through the benchmark tool, which names its 6 operations with its own verbs — this one among them, under a name only the door declares. describe explains any of them:
curl -X POST https://api.hanzo.ai/v1/mcp \
-H "Content-Type: application/json" \
-d '{
"jsonrpc": "2.0",
"id": 1,
"method": "tools/call",
"params": {
"name": "describe",
"arguments": {
"op": "get_benchmark_catalog"
}
}
}'How is this guide?