Hanzo

LLM Gateway

Unified proxy for 200+ LLM providers. One API, all models. Load balancing, caching, rate limiting, and observability.

API reference · Hanzo AI — Inference API → — every endpoint, generated from the OpenAPI spec.

LLM Gateway

One API for 200+ language models from every major provider. OpenAI-compatible interface with load balancing, caching, rate limiting, fallbacks, and full observability.

curl https://llm.hanzo.ai/v1/chat/completions \
  -H "Authorization: Bearer $HANZO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet-4-5-20250929",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

Why Hanzo LLM Gateway?

  • 200+ Models — Claude, GPT, Gemini, Llama, Mistral, and more
  • OpenAI Compatible — Drop-in replacement, use any OpenAI SDK
  • Smart Routing — Automatic fallbacks, load balancing, model selection
  • Cost Control — Per-key budgets, rate limits, usage analytics
  • Caching — Semantic and exact caching to reduce costs up to 90%
  • Observability — Full request logging, latency tracking, token analytics

Supported Providers

ProviderModelsFeatures
AnthropicClaude 4.x, Claude 3.xVision, tool use, extended context
OpenAIGPT-4o, o3, o4-miniFunction calling, vision, DALL-E
GoogleGemini 2.x, PaLMMultimodal, grounding
MetaLlama 3.x, Llama 4Open source, self-hosted
MistralMistral Large, CodestralEuropean, code generation
Together AI50+ open modelsFast inference, fine-tuning
GroqLlama, MixtralFastest inference
Zen LM600M-480BFrontier open-weight models

SDK Usage

Python

from openai import OpenAI

client = OpenAI(
    api_key="your-hanzo-api-key",
    base_url="https://llm.hanzo.ai/v1"
)

response = client.chat.completions.create(
    model="claude-sonnet-4-5-20250929",
    messages=[{"role": "user", "content": "Explain quantum computing"}]
)
print(response.choices[0].message.content)

TypeScript

import OpenAI from 'openai'

const client = new OpenAI({
  apiKey: process.env.HANZO_API_KEY,
  baseURL: 'https://llm.hanzo.ai/v1',
})

const completion = await client.chat.completions.create({
  model: 'claude-sonnet-4-5-20250929',
  messages: [{ role: 'user', content: 'Explain quantum computing' }],
})
console.log(completion.choices[0].message.content)

Key Features

Smart Routing

Automatic fallbacks between providers when one is down

Cost Management

Per-key budgets, rate limits, and usage analytics

Semantic Caching

Cache similar requests to reduce costs up to 90%

Guardrails

Content filtering, PII detection, and safety controls

Observability

Full request logging with latency and token analytics

Fine-tuning

Custom model training via Together AI and Zen LM

Configuration

model_list:
  - model_name: "default"
    litellm_params:
      model: "anthropic/claude-sonnet-4-5-20250929"
      api_key: "os.environ/ANTHROPIC_API_KEY"

  - model_name: "default"
    litellm_params:
      model: "openai/gpt-4o"
      api_key: "os.environ/OPENAI_API_KEY"

  - model_name: "fast"
    litellm_params:
      model: "groq/llama-3.1-70b"
      api_key: "os.environ/GROQ_API_KEY"

router_settings:
  routing_strategy: "latency-based-routing"
  fallbacks:
    - default: ["fast"]

API Endpoints

EndpointDescription
POST /v1/chat/completionsChat completions (streaming supported)
POST /v1/completionsText completions
POST /v1/embeddingsText embeddings
POST /v1/images/generationsImage generation
GET /v1/modelsList available models
POST /key/generateCreate API keys with budgets
GET /key/infoKey usage and budget info

Getting Started

Create an account, get an API key, and make your first Hanzo API call in under five minutes.

Run Your Org

From signup to a running brain, team, and bot for your organization — under ten minutes.

Authentication

The one way to authenticate against Hanzo IAM — canonical OIDC endpoints, the @hanzo/iam SDK, and per-framework integration.

API Keys

Key types, scopes, creation, rotation, and production best practices for Hanzo API keys.

Organizations

Multi-org setup, resource scoping, team management, and white-label login for Hanzo organizations.

Platform

Hanzo Cloud Platform — 40+ unified services for AI, automation, infrastructure, and operations.

API

The REST API reference for every Hanzo Cloud service -- one base URL, one bearer key, at api.hanzo.ai/v1.

SDKs

Official Hanzo SDKs for Python, TypeScript, Go, Rust, and C/C++.

Gateway

The unified, gated, priced API gateway for Hanzo Cloud at api.hanzo.ai -- one entry point, one key, every service.

Console

The unified cloud console for Hanzo Cloud -- manage every product, key, and organization from one app at console.hanzo.ai.

Integrations

Connect blob storage, Slack, Mixpanel, and Insights to your project.

Integrations & Extensions

Install and connect Hanzo across your browser, editor, desktop, docs, design tools, and business apps — one API key, one identity, everywhere.

Hanzo Studio

Visual AI engine for building image, video, 3D, and audio workflows. Node-based editor with GPU scaling, multi-tenant isolation, and enterprise billing.

AI Studio

Hanzo AI Studio — build, test, and ship AI apps and pipelines visually.

MCP

Model Context Protocol as a Service — Launch any MCP server with one command. 260+ tools available.

Hanzo Dev

AI-powered coding assistant for your terminal. Install, configure, and operate Hanzo Dev.

ZAP Protocol

Zero-Copy App Proto — High-performance Cap'n Proto RPC for AI agent communication. 10-100x faster than MCP with post-quantum security.

Agents

Build, deploy, and run autonomous agents — personas with a model, tools, guardrails, and shared memory.

Prompts

Versioned prompts with labels and history — manage, ship, and roll back prompts through one API.

Hanzo Embeddings

Generate, store, and search vector embeddings at scale — an OpenAI-compatible embeddings API on the Hanzo gateway that pairs with Vector.

Marketplace

Browse and deploy AI models and providers — real pricing and live availability.

Hanzo SQL

Managed PostgreSQL — provision databases, create copy-on-write branches, and scale reads with replicas over a single API and the standard Postgres wire protocol.

Hanzo Vector

Managed vector database for embeddings and semantic search — HNSW indexing, metadata filtering, and hybrid search, provisioned in one API call.

Hanzo KV

Managed key-value store — a Redis/Valkey-compatible cache, queue, and pub/sub engine with sub-millisecond reads, provisioned in one API call.

Hanzo Search

Managed search — full-text and hybrid indexes with typo tolerance, faceted filtering, and sub-50ms queries, provisioned in one API call.

Hanzo Datastore

Managed column-oriented analytics store — run sub-second SQL over billions of rows with a ClickHouse-compatible OLAP engine.

Hanzo DocDB

Managed document database — a MongoDB-compatible store with BSON documents, aggregation pipelines, and secondary indexes, backed by PostgreSQL.

Hanzo Storage

S3-compatible object storage — durable, versioned buckets for models, datasets, and media that work with any S3 client or SDK.

Hanzo Base

Multi-tenant Hanzo Base — provision and manage tenant backend instances, each a single-binary BaaS with REST, auth, realtime, and file storage.

Hanzo Memory

Your personal memory — a searchable, editable long-term memory service for agents and apps, built on semantic vector search.

GPUs

On-demand H100/H200/A100/L40S GPU compute — metered by the hour, provisioned from the console or API, billed to your org's cloud-usage ledger.

Machines

Compute machines and capacity across regions — the nodes of your clusters.

Edge

Compute at the edge — on-device AI inference close to your users, offline and private.

Hanzo Functions

Serverless compute platform

Pipelines

CI/CD pipelines from commit to deploy — build, push, and reconcile your fleet.

Hanzo Registry

Container images and artifacts — a self-hosted, S3-backed OCI registry at registry.hanzo.ai, mirrored to ghcr.io/hanzoai.

Networks

Blockchain networks — chain, nodes, status, and RPC endpoints.

Blockchain

On-chain settlement and networks — chain, RPC, and block explorer.

Hanzo O11y

Full-stack observability platform

Metrics

Product metrics, events, and sessions — OpenTelemetry- and Prometheus-compatible time series for your workloads.

Logs

Structured logs across all services — OpenTelemetry-compatible, correlated to traces by trace id.

Traces

Distributed traces across all services — OpenTelemetry-compatible, correlated with logs, metrics, and scores.

Sessions

Traces grouped into multi-turn sessions — one conversation or agent run across many requests.

Dashboards

Product analytics and observability dashboards for Hanzo Cloud -- usage, cost, latency, and health in one view.

Datasets

Curate evaluation datasets and items — the input/expected-output pairs your experiments run against.

Experiments

Dataset runs, comparisons, and experiment analytics — score a dataset against a model and compare runs.

Scores

Evaluation scores from feedback, graders, and review — attached to traces, observations, and sessions.

Score Configs

Score definitions — data types, ranges, and categories that every score conforms to.

Annotation Queues

Review queues for scoring traces and observations against a defined set of score configs.

IAM

Organizations, users, and roles (RBAC) for Hanzo Cloud, powered by Hanzo IAM and the hanzo.id OIDC issuer.

KMS

Encryption keys, secrets, and cryptographic operations for Hanzo Cloud -- the secrets control plane, Hanzo KMS.

Authz

Fine-grained authorization policies and access checks for Hanzo Cloud -- model-based access control powered by Hanzo Authz.

MPC

Threshold signing and multi-party computation for Hanzo Cloud -- split-key custody with post-quantum finality, powered by Hanzo MPC.

Zero Trust

Private service access for Hanzo Cloud -- routers, identities, policies, and sessions over an identity-based mTLS overlay.

Hanzo Zero Trust

Programmable zero-trust networking

Overview

Hanzo Commerce - Full-stack e-commerce platform with AI-powered recommendations

Billing

Usage, credits, invoices, and payment methods for Hanzo Cloud -- usage-based metering, cloud credits, and on-chain HUSD top-up.

Referrals

Your referral link, invite history, and cloud-credit earnings on Hanzo Cloud -- refer developers, earn credits and revenue share.

Hanzo Tasks

Durable workflow execution engine

Hanzo PubSub

High-performance messaging and streaming

Crawl

Crawl and extract the web for your agents — a Firecrawl-compatible API.

Research Papers

Technical whitepapers on Hanzo AI infrastructure, the Zen frontier model family, post-quantum cryptography, and defense applications.

How is this guide?

Last updated on

On this page