Datasets

A dataset is a curated collection of items — each an input paired with an expected output — used to evaluate models and agents. Datasets are the fixed yardstick an experiment measures against. Part of the Hanzo Cloud evaluation surface, tenant-scoped by org.

Create a dataset

curl -X POST https://api.hanzo.ai/v1/evals/datasets \
  -H "Authorization: Bearer sk-..." \
  -H "Content-Type: application/json" \
  -d '{
    "name": "support-qa",
    "description": "Golden answers for common support questions"
  }'

Add items

Each item carries an input and an expectedOutput; metadata is free-form and travels with the item into every run.

Items are versioned by dataset, so a run always evaluates against a known snapshot.

curl https://api.hanzo.ai/v1/evals/datasets \
  -H "Authorization: Bearer sk-..."

Run against a model

A dataset becomes useful when you run it. Pass the dataset name to an experiment — POST /v1/evals/runs — to score every item against a model, optionally with an LLM judge.

Related

• Experiments — dataset runs, comparisons, and analytics
• Scores — the per-item results a run produces
• Prompts — version the prompt an experiment evaluates