Hanzo
Hanzo Skills Reference

Hanzo Operator

Hanzo Operator is a unified Kubernetes operator that manages all Hanzo production infrastructure declaratively via 7 CRDs under the `hanzo.ai/v1alpha1` API group. Built with Kubebuilder v4 and cont...

Overview

Hanzo Operator is a unified Kubernetes operator that manages all Hanzo production infrastructure declaratively via 7 CRDs under the hanzo.ai/v1alpha1 API group. Built with Kubebuilder v4 and controller-runtime, it reconciles high-level Hanzo resource specs into the full set of underlying K8s objects (Deployments, StatefulSets, Services, Ingress, HPA, PDB, NetworkPolicy, CronJobs, KMSSecret CRs).

Why Hanzo Operator?

  • Declarative infrastructure: Define an entire Hanzo service, datastore, or network in one CR
  • 7 CRDs: HanzoService, HanzoDatastore, HanzoGateway, HanzoMPC, HanzoNetwork, HanzoIngress, HanzoPlatform
  • Idempotent reconciliation: Uses ctrl.CreateOrUpdate with MutateFuncFor for safe convergence
  • KMS integration: Automatically creates kms.hanzo.ai/v1alpha1 KMSSecret CRs for secret management
  • Owner references: Full GC cascade -- deleting the parent CR cleans up all children
  • Phase lifecycle: Pending, Creating, Running, Degraded, Deleting

Tech Stack

  • Language: Go 1.26
  • Framework: Kubebuilder v4, controller-runtime v0.23.1
  • K8s API: k8s.io/* v0.35.0
  • Image: ghcr.io/hanzoai/operator:latest
  • Namespace: hanzo-operator-system
  • Stats: 28 Go source files, ~7,400 lines, 13,022 lines CRD YAML

When to use

  • Deploying Hanzo services to Kubernetes declaratively
  • Managing API gateways (KrakenD-based) via HanzoGateway
  • Provisioning datastores (PostgreSQL, Redis, MongoDB) via HanzoDatastore
  • Running blockchain validator networks via HanzoNetwork
  • Managing multi-party computation infrastructure via HanzoMPC
  • Orchestrating a full Hanzo platform stack via HanzoPlatform

Hard requirements

  1. Kubernetes v1.11.3+ cluster with CRD support
  2. cert-manager for TLS (used by HanzoIngress)
  3. KMS operator (kms.hanzo.ai) for KMSSecret reconciliation
  4. RBAC permissions: core, apps, autoscaling, batch, networking, policy, hanzo.ai/, kms.hanzo.ai/

Quick reference

ItemValue
LanguageGo 1.26
FrameworkKubebuilder v4, controller-runtime v0.23.1
API Grouphanzo.ai/v1alpha1
CRD Count7
Imageghcr.io/hanzoai/operator:latest
Namespacehanzo-operator-system
Repogithub.com/hanzoai/operator
Branchmain
LicenseApache 2.0

CRD Reference

CRDShort NameCreates
HanzoServicehsvcDeployment, Service, Ingress, HPA, PDB, NetworkPolicy, KMSSecret
HanzoDatastorehdsStatefulSet, headless Service, PVC, CronJob (backup), KMSSecret
HanzoGatewayhgwDeployment, Service, ConfigMap (KrakenD config), Ingress
HanzoMPChmpcStatefulSet, headless Service, Dashboard Deployment, Cache Deployment, Ingress
HanzoNetworkhnetStatefulSet (validators), Deployments (bootnode/indexer/explorer/bridge), Services, ConfigMaps
HanzoIngresshingMultiple Ingress resources with cert-manager TLS
HanzoPlatformhplatChild CRDs (composes all of the above)

Repository Structure

api/v1alpha1/
  common_types.go              # Shared types (ContainerSpec, IngressSpec, etc.)
  hanzoservice_types.go        # HanzoService CRD spec
  hanzodatastore_types.go      # HanzoDatastore CRD spec
  hanzogateway_types.go        # HanzoGateway CRD spec
  hanzompc_types.go            # HanzoMPC CRD spec
  hanzonetwork_types.go        # HanzoNetwork CRD spec
  hanzoingress_types.go        # HanzoIngress CRD spec
  hanzoplatform_types.go       # HanzoPlatform CRD spec
  types.go                     # Additional shared types
  zz_generated.deepcopy.go     # Generated DeepCopy methods
cmd/
  main.go                      # Manager entry point
internal/
  controller/
    hanzoservice_controller.go   # HanzoService reconciler
    hanzodatastore_controller.go # HanzoDatastore reconciler
    hanzogateway_controller.go   # HanzoGateway reconciler
    hanzompc_controller.go       # HanzoMPC reconciler
    hanzonetwork_controller.go   # HanzoNetwork reconciler
    hanzoingress_controller.go   # HanzoIngress reconciler
    hanzoplatform_controller.go  # HanzoPlatform reconciler
    predicates.go                # Event filters (createOrUpdatePred, etc.)
    helpers.go                   # Shared controller utilities
  manifests/                     # K8s object builders (builder, labels, mutate)
  status/                        # Condition management
  metrics/                       # Prometheus metrics
  config/                        # Feature gates
config/
  crd/bases/                     # Generated CRD YAML (13k lines)
  rbac/                          # ClusterRole, bindings
  manager/                       # Deployment template
  samples/                       # Sample CRs for all 7 CRDs
Dockerfile                       # Multi-stage Go build (Alpine)
Makefile                         # Build, test, deploy targets

Build and Deploy

# Regenerate CRDs and DeepCopy
make manifests generate

# Build local binary
make build

# Run tests
make test

# Build Docker image
make docker-build IMG=ghcr.io/hanzoai/operator:latest

# Build and push (multi-platform)
docker buildx build --platform linux/amd64 --push -t ghcr.io/hanzoai/operator:latest .

# Install CRDs into cluster
make install

# Deploy operator
make deploy IMG=ghcr.io/hanzoai/operator:latest

# Apply sample CRs
kubectl apply -k config/samples/

Production Deployment

Universe manifests at ~/work/hanzo/universe/infra/k8s/hanzo-operator/:

kubectl apply -k universe/infra/k8s/hanzo-operator/

Key Patterns

  • Predicate filtering: createOrUpdatePred, updateOrDeletePred, statusChangePred control which events trigger reconciliation
  • CreateOrUpdate with MutateFuncFor: Idempotent object reconciliation -- creates if missing, patches if changed
  • Owner references: All child objects have owner refs for automatic garbage collection
  • Phase lifecycle: Resources transition through Pending, Creating, Running, Degraded, Deleting
  • KMSSecret delegation: Operator creates kms.hanzo.ai/v1alpha1 KMSSecret CRs; the existing KMS operator handles actual secret sync from kms.hanzo.ai

Troubleshooting

IssueCauseSolution
RBAC errors on deployMissing cluster-adminGrant cluster-admin or use make deploy
CRD not foundCRDs not installedmake install first
KMSSecret not syncingKMS operator not runningDeploy KMS operator to cluster
Reconcile loopSpec driftCheck object mutate functions in manifests/
Image pull errorsGHCR authEnsure imagePullSecrets configured
  • hanzo/hanzo-charts.md - Helm charts (alternative to operator CRDs)
  • hanzo/hanzo-universe.md - Universe manifests (where operator is deployed)
  • hanzo/hanzo-kms.md - KMS for secret management (KMSSecret CRs)
  • hanzo/hanzo-platform.md - PaaS platform (managed by HanzoPlatform CRD)

How is this guide?

Last updated on

On this page