Hanzo AI API

Hanzo's inference API. Serves the Zen model family at api.hanzo.ai/v1, with cost tracking, budgets, and fallback routing. The server behind it is open source, so you can run it yourself.

Overview

The Hanzo AI API is Hanzo's inference API. It serves the Zen model family at api.hanzo.ai/v1, tracks cost per request, enforces budgets, and fails over between backends. The server is open source (github.com/hanzoai/llm), so you can also run it inside your own network against backends you choose.

Why Hanzo AI API?

Zen models: enso, zen5, zen5-coder, zen-vl, zen-embedding and the rest of the family, served directly
Cost tracking: Per-request attribution via Hanzo Console
Budgets and rate limits: Spend caps and concurrency limits per key
Fallback routing: Retry on a second backend when the first returns 429, 500, or times out
Self-hostable: Run it behind your firewall with your own keys

Open source

Repo: github.com/hanzoai/llm.

When to use

Calling Zen models over HTTP
Running centralized model access for your team
Cost tracking and budget enforcement
Self-hosting model access behind your firewall
Adding your own models or backends

Hard requirements

An API key — HANZO_API_KEY for api.hanzo.ai, or a key for whichever backend you point a self-hosted instance at
Port 4000 available (default, self-hosted)
PostgreSQL for logging (optional)

Quick reference

Item	Value
Public endpoint	`https://api.hanzo.ai/v1`
Internal endpoint	`http://llm.hanzo.svc.cluster.local:4000/v1`
Port	4000
Config	`config.yaml` or env vars
Dashboard	`https://console.hanzo.ai`
Repo	`github.com/hanzoai/llm`

First request

curl https://api.hanzo.ai/v1/chat/completions \
  -H "Authorization: Bearer ${HANZO_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "zen5",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Zen models

Zen is Hanzo's own model family, trained by Hanzo and served on this API. Current ids:

Id	Use
`enso`, `enso-flash`, `enso-ultra`	General reasoning, three sizes
`zen5`, `zen5-pro`, `zen5-flash`	General chat, three sizes
`zen5-coder`	Code
`zen-vl`	Vision-language
`zen-embedding`, `zen-rerank`	Retrieval
`zen-image`, `zen-video`, `zen-voice`, `zen-music`, `zen-foley`	Generation
`zen-guard`	Safety classification
`best`	Routes to the strongest model available for the request

See hanzo/zenlm.md for architecture and local inference.

Self-hosting

Docker

docker run -d --name hanzo-llm \
  -p 4000:4000 \
  -e HANZO_MASTER_KEY="${HANZO_MASTER_KEY}" \
  -e HANZO_API_KEY="${HANZO_API_KEY}" \
  ghcr.io/hanzoai/llm:latest

Config file

A self-hosted instance serves whatever you put in model_list. Point entries at your own inference servers, at outside providers using keys you supply, or at both.

# config.yaml
model_list:
  # Your own inference server (Hanzo Engine, vLLM, Ollama, ...)
  - model_name: local
    llm_params:
      model: openai/local
      api_base: http://engine.hanzo.svc.cluster.local:8000/v1
      api_key: os.environ/HANZO_ENGINE_KEY

  # Outside providers, with keys you supply
  - model_name: gpt-4o
    llm_params:
      model: openai/gpt-4o
      api_key: os.environ/OPENAI_API_KEY

  - model_name: claude-sonnet
    llm_params:
      model: anthropic/claude-sonnet-4-20250514
      api_key: os.environ/ANTHROPIC_API_KEY

router_settings:
  routing_strategy: least-busy
  num_retries: 3
  fallbacks:
    - local: [gpt-4o, claude-sonnet]

general_settings:
  master_key: os.environ/HANZO_MASTER_KEY
  database_url: os.environ/DATABASE_URL

Zen models are served from api.hanzo.ai/v1 — a self-hosted instance reaches them the same way any other client does, with your HANZO_API_KEY.

Make commands

# Clone from github.com/hanzoai first
cd <project>
make dev              # Start dev server (port 4000)
make up               # Docker compose up
docker compose up -d  # Alternative

Test request

Against a fresh self-hosted instance, the master key is also a valid caller key:

curl http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer ${HANZO_MASTER_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "local",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Core Concepts

Routing

Client Request
    |
    v
+------------------+
|  AI API          |
|  (port 4000)     |
+------------------+
| Router:          |
| +- least-busy    |-->  Backend A
| +- fallback      |-->  Backend B
| +- cost-based    |-->  Backend C
+------------------+    ^
| Logging:         |    |
| +- PostgreSQL    |    |
| +- Console       |----+ (cost tracking)
+------------------+

Fallback Chains

When a backend fails (429, 500, timeout), the router retries against the next entry:

fallbacks:
  - local: [gpt-4o, claude-sonnet]
  - gpt-4o: [claude-sonnet]

Budget & Rate Limits

general_settings:
  max_budget: 100.00           # USD per month
  budget_duration: 1m          # Reset monthly
  max_parallel_requests: 100   # Concurrent limit

Production Deployment

Kubernetes

apiVersion: apps/v1
kind: Deployment
metadata:
  name: hanzo-llm
spec:
  replicas: 3
  template:
    spec:
      containers:
      - name: llm
        image: ghcr.io/hanzoai/llm:latest
        ports:
        - containerPort: 4000
        env:
        - name: HANZO_MASTER_KEY
          valueFrom:
            secretKeyRef:
              name: llm-secrets
              key: master-key
        - name: DATABASE_URL
          valueFrom:
            secretKeyRef:
              name: llm-secrets
              key: database-url

HTTP clients

/v1/chat/completions takes and returns the chat-completions JSON shape, so an HTTP client already written against that shape works once its base URL points at https://api.hanzo.ai/v1 and it sends your HANZO_API_KEY.

Troubleshooting

Issue	Cause	Solution
401 on requests	Wrong master key	Check `HANZO_MASTER_KEY`
Backend timeout	Upstream backend slow	Increase timeout or add a fallback
Cost not tracking	No DATABASE_URL	Add PostgreSQL connection
Model not found	Not in config	Add to `config.yaml`

hanzo/hanzo-chat.md - Chat UI (calls this API)
hanzo/hanzo-console.md - Observability (receives cost data)
hanzo/python-sdk.md - Client library
hanzo/zenlm.md - Zen model family

On this page