Smart routing

Set model: "auto" on a Hanzo Cloud chat completion and each prompt is routed to the model that fits the task and your service-level objective — a local Zen model, a cheap tier, or a frontier model. You are billed as whatever actually served the request, and every response carries an X-Routed-Model header so the choice is transparent.

The value is cost. Most prompts in a real workload are easy and never need a frontier model. Routing the easy ones to on-device or open models — and reserving frontier models for the hardest prompts — can cut API spend up to 90%. The exact number depends on your prompt mix; see Honest savings below.

Enable routing

router:
  enabled: true        # resolve model "auto" to a concrete model
  endpoint: ""         # the learned Enso engine; empty routes by the rule table alone
  cost_ceiling: 0.0    # default cost cap, USD per 1K tokens
  prefer:              # task class -> ordered model ids; the first servable wins
    code: [zen5-coder, zen5-pro]
    default: [zen5]

Send model: "auto"

Set the model to auto on a chat completion against api.hanzo.ai. Nothing else changes — same request shape, same response shape.

curl https://api.hanzo.ai/v1/chat/completions \
  -H "Authorization: Bearer $HANZO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [
      { "role": "user", "content": "Summarize this changelog in one line." }
    ]
  }' -i

import { Hanzo } from '@hanzo/ai'

const hanzo = new Hanzo({ secretKey: process.env.HANZO_API_KEY })

const res = await hanzo.chat.completions.create({
  model: 'auto',
  messages: [
    { role: 'user', content: 'Summarize this changelog in one line.' },
  ],
})

// The model that actually served the request:
console.log(res.model) // e.g. "zen5-flash" | "zen5" | a frontier model

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["HANZO_API_KEY"],
    base_url="https://api.hanzo.ai/v1",
)

res = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Summarize this changelog in one line."}],
)

print(res.model)  # the model that actually served the request

curl -i prints the response headers, so you can read the X-Routed-Model value the router picked. From an SDK the same answer is on the response body's model field.

Steering with an SLO

Routing is automatic, but two optional headers bound it per request without changing the body. The router still picks within them:

curl https://api.hanzo.ai/v1/chat/completions \
  -H "Authorization: Bearer $HANZO_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Max-Cost: 0.5" \
  -H "X-Max-Latency-Ms: 2000" \
  -d '{"model": "auto", "messages": [{"role": "user", "content": "Summarize this changelog."}]}'

An org-level cost ceiling fills X-Max-Cost when you do not send it; a header you send wins when it is lower. Omit both to let the router balance cost and quality on its own.

The X-Routed-Model header

Every routed response tells you what served it. Read the model id from either the X-Routed-Model response header or the model field in the JSON body — they always match. This makes routing fully auditable: log the header to see, per request, exactly which tier answered and what you were billed for.

HTTP/1.1 200 OK
X-Routed-Model: zen5-flash
Content-Type: application/json

Per-surface toggles

Smart routing is a Hanzo Cloud feature, so it is available everywhere Hanzo runs. Turn it on per surface:

• Console — platform.hanzo.ai/ai-accounts → Smart routing
• hanzo.chat — Settings → Model → Auto (smart routing)
• Hanzo Desktop — Settings → Model → Auto
• hanzo.app — Settings → Model → Auto
• Mobile — Settings → Model → Auto

Selecting Auto on any surface is equivalent to sending model: "auto" on the API.

Rules underneath, learned on top

The router is Enso. Each request is classified into a task class, and an org's strategy decides who picks within it: enso, the default, lets the learned policy lead and falls back to the rule table when it is unsure or unreachable; heuristic uses the rule table alone, for routing you can audit line by line. A per-task override pins a model ahead of both. The request contract is the same under every strategy — model: "auto", X-Routed-Model and the SLO headers — so you can move between them without touching client code. zen-router is accepted as an alias for auto.

Rewards train the learned policy: send POST /v1/ai/feedback with the response id and what the user did with the answer.

Honest savings

The savings are real but workload-dependent, so we frame them honestly:

• On a worked mix of 70% simple prompts routed to local/open models (~$0), 20% to a cheap tier, and 10% to a frontier model, blended API spend drops ~89% versus sending everything to a single frontier model. Your mix determines your number — we say "up to 90%," never a guarantee.
• On the public RouterBench dataset, oracle routing reaches 0.909 quality at roughly 13× lower cost than the best single model — cited as a replication smoke run, not a Hanzo-specific benchmark.
• Full total-cost-of-ownership also improves from seat consolidation, overage avoidance, and running on-device Zen models where they suffice.

We never publish fabricated benchmarks. The worked mix above is an illustration, not a measurement; the RouterBench figure is a public third-party result. Hanzo's own measurements of the learned policy are in the Hanzo Router paper.