# Smart routing

URL: https://docs.hanzo.ai/docs/usage/routing

> Send model "auto" to api.hanzo.ai and each prompt is routed to the model that fits the task and your SLO — for up to 90% lower AI spend.

Set `model: "auto"` on a Hanzo Cloud chat completion and each prompt is routed
to the model that fits the task and your service-level objective — a local
[Zen](https://huggingface.co/zenlm) model, a cheap tier, or a frontier model.
You are billed as whatever actually served the request, and every response
carries an `X-Routed-Model` header so the choice is transparent.

The value is cost. Most prompts in a real workload are easy and never need a
frontier model. Routing the easy ones to on-device or open models — and
reserving frontier models for the hardest prompts — can cut API spend
\*\*up to 90%\*\*. The exact number depends on your prompt mix; see
[Honest savings](https://docs.hanzo.ai/docs/usage/routing#honest-savings) below.

## Enable routing

### Cloud — toggle in the console

On [platform.hanzo.ai](https://platform.hanzo.ai), open **AI Accounts**
(`/ai-accounts`) and turn on **Smart routing**. That is all cloud users need —
requests with `model: "auto"` start routing immediately.

### Self-hosted — set `router.enabled`

If you run the gateway ([hanzoai/ai](https://github.com/hanzoai/ai)) yourself,
enable the router in its model config:

```yaml title="conf/models.yaml"
router:
  enabled: true        # resolve model "auto" to a concrete model
  endpoint: ""         # the learned Enso engine; empty routes by the rule table alone
  cost_ceiling: 0.0    # default cost cap, USD per 1K tokens
  prefer:              # task class -> ordered model ids; the first servable wins
    code: [zen5-coder, zen5-pro]
    default: [zen5]
```

Restart the gateway to pick up the change.

## Send `model: "auto"`

Set the model to `auto` on a chat completion against `api.hanzo.ai`. Nothing
else changes — same request shape, same response shape.

**curl**

```bash
curl https://api.hanzo.ai/v1/chat/completions \
  -H "Authorization: Bearer $HANZO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [
      { "role": "user", "content": "Summarize this changelog in one line." }
    ]
  }' -i
```

**TypeScript**

```typescript
import { Hanzo } from '@hanzo/ai'

const hanzo = new Hanzo({ secretKey: process.env.HANZO_API_KEY })

const res = await hanzo.chat.completions.create({
  model: 'auto',
  messages: [
    { role: 'user', content: 'Summarize this changelog in one line.' },
  ],
})

// The model that actually served the request:
console.log(res.model) // e.g. "zen5-flash" | "zen5" | a frontier model
```

**Python**

```python
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["HANZO_API_KEY"],
    base_url="https://api.hanzo.ai/v1",
)

res = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Summarize this changelog in one line."}],
)

print(res.model)  # the model that actually served the request
```

`curl -i` prints the response headers, so you can read the `X-Routed-Model`
value the router picked. From an SDK the same answer is on the response body's
`model` field.

## Steering with an SLO

Routing is automatic, but two optional headers bound it per request without
changing the body. The router still picks within them:

```bash
curl https://api.hanzo.ai/v1/chat/completions \
  -H "Authorization: Bearer $HANZO_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Max-Cost: 0.5" \
  -H "X-Max-Latency-Ms: 2000" \
  -d '{"model": "auto", "messages": [{"role": "user", "content": "Summarize this changelog."}]}'
```

| Header             | Meaning                                                    |
| ------------------ | ---------------------------------------------------------- |
| `X-Max-Cost`       | Cost ceiling in USD per 1K tokens; excludes pricier models |
| `X-Max-Latency-Ms` | Latency ceiling in milliseconds; excludes slower models    |

An org-level cost ceiling fills `X-Max-Cost` when you do not send it; a header
you send wins when it is lower. Omit both to let the router balance cost and
quality on its own.

## The `X-Routed-Model` header

Every routed response tells you what served it. Read the model id from either
the `X-Routed-Model` response header or the `model` field in the JSON body —
they always match. This makes routing fully auditable: log the header to see,
per request, exactly which tier answered and what you were billed for.

```http
HTTP/1.1 200 OK
X-Routed-Model: zen5-flash
Content-Type: application/json
```

## Per-surface toggles

Smart routing is a Hanzo Cloud feature, so it is available everywhere Hanzo
runs. Turn it on per surface:

- **Console** — [platform.hanzo.ai/ai-accounts](https://platform.hanzo.ai/ai-accounts) → **Smart routing**
- **hanzo.chat** — Settings → Model → \*_Auto (smart routing)_\*
- **Hanzo Desktop** — Settings → Model → **Auto**
- **hanzo.app** — Settings → Model → **Auto**
- **Mobile** — Settings → Model → **Auto**

Selecting **Auto** on any surface is equivalent to sending `model: "auto"` on
the API.

## Rules underneath, learned on top

The router is [Enso](https://docs.hanzo.ai/docs/models/enso). Each request is classified into a task
class, and an org's strategy decides who picks within it: `enso`, the default,
lets the learned policy lead and falls back to the rule table when it is unsure
or unreachable; `heuristic` uses the rule table alone, for routing you can audit
line by line. A per-task override pins a model ahead of both. The request
contract is the same under every strategy — `model: "auto"`, `X-Routed-Model`
and the SLO headers — so you can move between them without touching client
code. `zen-router` is accepted as an alias for `auto`.

Rewards train the learned policy: send
[`POST /v1/ai/feedback`](https://docs.hanzo.ai/docs/models/enso#teach-it) with the response `id` and
what the user did with the answer.

## Honest savings

The savings are real but workload-dependent, so we frame them honestly:

- On a worked mix of **70% simple** prompts routed to local/open models (\~$0),
  \*_20%_\* to a cheap tier, and \*_10%_\* to a frontier model, blended API spend
  drops \*\*\~89%\*\* versus sending everything to a single frontier model. Your
  mix determines your number — we say "up to 90%," never a guarantee.
- On the public **RouterBench** dataset, oracle routing reaches **0.909**
  quality at roughly **13× lower cost** than the best single model — cited as a
  replication smoke run, not a Hanzo-specific benchmark.
- Full total-cost-of-ownership also improves from seat consolidation, overage
  avoidance, and running on-device Zen models where they suffice.

We never publish fabricated benchmarks. The worked mix above is an illustration,
not a measurement; the RouterBench figure is a public third-party result.
Hanzo's own measurements of the learned policy are in the
[Hanzo Router paper](https://github.com/hanzoai/papers/tree/main/hanzo-router).

- [Enso](https://docs.hanzo.ai/docs/models/enso): How the router picks, how to bound it, how to teach it