Smart routing

Set model: "auto" on a Hanzo Cloud chat completion and each prompt is routed to the model that fits the task and your service-level objective — a local zen model, a cheap tier, or a frontier model. You are billed as whatever actually served the request, and every response carries an X-Routed-Model header so the choice is transparent.

The value is cost. Most prompts in a real workload are easy and never need a frontier model. Routing the easy ones to on-device or open models — and reserving frontier models for the hardest prompts — can cut API spend up to 90%. The exact number depends on your prompt mix; see Honest savings below.

Enable routing

router:
  enabled: true          # route model:"auto" requests
  strategy: heuristic    # built-in heuristic (default) | zen-router

Send model: "auto"

Point any OpenAI-compatible client at api.hanzo.ai and set the model to auto. Nothing else changes — same request shape, same response shape.

curl https://api.hanzo.ai/v1/chat/completions \
  -H "Authorization: Bearer $HANZO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [
      { "role": "user", "content": "Summarize this changelog in one line." }
    ]
  }' -i

import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'https://api.hanzo.ai/v1',
  apiKey: process.env.HANZO_API_KEY,
});

const res = await client.chat.completions.create({
  model: 'auto',
  messages: [
    { role: 'user', content: 'Summarize this changelog in one line.' },
  ],
});

// The model that actually served the request:
console.log(res.model); // e.g. "zen-nano" | "zen5" | a frontier model

Steering with an SLO

Routing is automatic, but you can steer it with an optional slo object. The router honors your bounds while still choosing the cheapest model that clears them:

{
  "model": "auto",
  "slo": {
    "max_latency_ms": 2000,
    "min_quality": 0.85,
    "max_cost_per_1k": 0.50
  },
  "messages": [ ... ]
}

Omit slo entirely to let the router balance cost and quality on its own.

The X-Routed-Model header

Every routed response tells you what served it. Read the model id from either the X-Routed-Model response header or the model field in the JSON body — they always match. This makes routing fully auditable: log the header to see, per request, exactly which tier answered and what you were billed for.

HTTP/1.1 200 OK
X-Routed-Model: zen-nano
Content-Type: application/json

Per-surface toggles

Smart routing is a Hanzo Cloud feature, so it is available everywhere Hanzo runs. Turn it on per surface:

• Console — console.hanzo.ai/ai-accounts → Smart routing
• hanzo.chat — Settings → Model → Auto (smart routing)
• Hanzo Desktop — Settings → Model → Auto
• hanzo.app — Settings → Model → Auto
• Mobile — Settings → Model → Auto

Selecting Auto on any surface is equivalent to sending model: "auto" on the API.

Heuristic today, zen-router next

Routing ships with a built-in heuristic — it scores each prompt's difficulty and matches it to a tier. It works out of the box with no model to host and no extra latency.

For higher accuracy you can upgrade to the trained zen-router (zenlm/zen-router) by setting router.strategy: zen-router (self-hosted) or switching the strategy in the console (cloud). The request contract is identical — model: "auto", X-Routed-Model, and the slo fields all stay the same — so you can move between strategies without touching client code.

Honest savings

The savings are real but workload-dependent, so we frame them honestly:

• On a worked mix of 70% simple prompts routed to local/open models (~$0), 20% to a cheap tier, and 10% to a frontier model, blended API spend drops ~89% versus sending everything to a single frontier model. Your mix determines your number — we say "up to 90%," never a guarantee.
• On the public RouterBench dataset, oracle routing reaches 0.909 quality at roughly 13× lower cost than the best single model — cited as a replication smoke run, not a Hanzo-specific benchmark.
• Full total-cost-of-ownership also improves from seat consolidation, overage avoidance, and running on-device zen models where they suffice.

We never publish fabricated benchmarks. The worked mix above is an illustration, not a measurement; the RouterBench figure is a public third-party result. Hanzo has published no routing paper of its own, so there is none to cite here.