Hanzo
Hanzo Skills Reference

Hanzo PubSub - Event Streaming and Message Queue

Hanzo PubSub is a high-performance messaging system for pub/sub, persistent streams, and exactly-once delivery. It is a fork of NATS Server with Hanzo branding and a bundled Kafka-compatible Redpan...

Overview

Hanzo PubSub is a high-performance messaging system for pub/sub, persistent streams, and exactly-once delivery. It is a fork of NATS Server with Hanzo branding and a bundled Kafka-compatible Redpanda sidecar. Written in Go. Provides the messaging backbone for Hanzo infrastructure.

Why Hanzo PubSub?

  • Pub/Sub Messaging: Subject-based routing with wildcard subscriptions
  • JetStream: Persistent streams with configurable retention and replay
  • Consumer Groups: Scalable consumption with automatic load balancing
  • Exactly-Once Delivery: Deduplication and acknowledgment semantics
  • Built-in Key-Value Store: Distributed KV on top of JetStream
  • Built-in Object Store: Large object storage via JetStream
  • Kafka Compatible: Bundled Redpanda sidecar speaks Kafka wire protocol
  • Clustering: Horizontal scaling with automatic failover via Raft

Tech Stack

  • Language: Go (module: github.com/nats-io/nats-server/v2)
  • Go Version: 1.26
  • License: Apache 2.0
  • Kafka Sidecar: Redpanda v25.1.9

OSS Base

Fork of NATS Server. Repo: hanzoai/pubsub.

When to use

  • Inter-service messaging in Hanzo infrastructure
  • Durable event streaming with replay capability
  • Work queues with load-balanced consumers
  • Real-time notifications and event-driven architectures
  • Kafka-compatible streaming (via Redpanda sidecar)
  • Lightweight distributed key-value or object storage

Hard requirements

  1. Go 1.26+ to build from source
  2. Docker for container deployment
  3. Persistent volume for JetStream data durability

Quick reference

ItemValue
Repogithub.com/hanzoai/pubsub
Modulegithub.com/nats-io/nats-server/v2 (upstream path retained)
Go Version1.26
LicenseApache 2.0
Client Port4222
Monitoring Port8222
Cluster Port6222
WebSocket Port5222
Kafka Port9092 (Redpanda sidecar)
Docker Imagehanzoai/pubsub:latest
Binarypubsub (built from main.go)
Config/pubsub/conf/server.conf

One-file quickstart

Docker

docker run -d --name hanzo-pubsub \
  -p 4222:4222 \
  -p 8222:8222 \
  hanzoai/pubsub:latest

Docker Compose

# compose.yml
services:
  pubsub:
    image: hanzoai/pubsub:latest
    ports:
      - "4222:4222"   # Client connections
      - "8222:8222"   # HTTP monitoring
      - "6222:6222"   # Cluster routing
    volumes:
      - pubsub-data:/data
    command: ["--jetstream", "--store_dir=/data"]

  # Optional: Kafka-compatible sidecar
  kafka:
    build: kafka/
    ports:
      - "9092:9092"   # Kafka protocol
      - "8081:8081"   # Schema registry
      - "8082:8082"   # HTTP proxy
      - "9644:9644"   # Admin API

volumes:
  pubsub-data:

Build from source

git clone https://github.com/hanzoai/pubsub.git
cd pubsub
go build -o pubsub .
./pubsub --jetstream --store_dir /tmp/pubsub-data

Core Concepts

Architecture

┌──────────────────────────────────────────────────┐
│                    Cluster                        │
│  ┌─────────┐    ┌─────────┐    ┌─────────┐      │
│  │ Node 1  │<-->│ Node 2  │<-->│ Node 3  │      │
│  │ :4222   │    │ :4222   │    │ :4222   │      │
│  └─────────┘    └─────────┘    └─────────┘      │
│       │              │              │            │
│       └──────────────┼──────────────┘            │
│                      │                           │
│              ┌───────┴───────┐                   │
│              │  JetStream    │                   │
│              │  (Streams,    │                   │
│              │   KV, ObjStore)│                  │
│              └───────────────┘                   │
└──────────────────────────────────────────────────┘

       ┌───────────────────────┐
       │  Redpanda Sidecar     │
       │  (Kafka :9092)        │
       │  (Schema Reg :8081)   │
       └───────────────────────┘

Server Configuration

Default config at docker/nats-server.conf:

# Client port
port: 4222

# HTTP monitoring
monitor_port: 8222

# Clustering
cluster {
  port: 6222
  authorization {
    user: ruser
    password: T0pS3cr3t
    timeout: 2
  }
  routes = []
}

Port Allocation

PortProtocolPurpose
4222TCPClient connections (NATS protocol)
8222HTTPMonitoring and management
6222TCPCluster routing between nodes
5222WebSocketWebSocket client connections
9092TCPKafka protocol (Redpanda sidecar)
8081HTTPSchema registry (Redpanda)
8082HTTPHTTP proxy (Redpanda)
9644HTTPRedpanda admin API

Directory Structure

pubsub/
  main.go              # Entry point (configures and runs server)
  server/              # Core server implementation
  conf/                # Configuration parser (lexer, parser)
  internal/
    antithesis/        # Deterministic testing
    fastrand/          # Fast random number generation
    ldap/              # LDAP authentication
    ocsp/              # OCSP stapling
    testhelper/        # Test utilities
  kafka/
    Dockerfile         # Redpanda (Kafka-compatible) sidecar
  logger/              # Logging
  docker/
    nats-server.conf   # Default server configuration
    Dockerfile.nightly # Nightly build
  test/                # Integration tests
  scripts/             # Build and CI scripts

Client SDKs

LanguagePackage
Gogithub.com/nats-io/nats.go
Pythonnats-py (pip install nats-py)
TypeScriptnats (npm install nats)
Rustnats (cargo add nats)

All NATS clients work directly with Hanzo PubSub (wire-compatible).

Usage Examples

Basic Pub/Sub (Go):

nc, _ := nats.Connect("nats://localhost:4222")
defer nc.Close()

// Subscribe
nc.Subscribe("events.>", func(msg *nats.Msg) {
    fmt.Printf("Received: %s\n", string(msg.Data))
})

// Publish
nc.Publish("events.user.created", []byte(`{"user_id":"123"}`))

JetStream Persistent Streams (Go):

nc, _ := nats.Connect("nats://localhost:4222")
js, _ := nc.JetStream()

// Create stream
js.AddStream(&nats.StreamConfig{
    Name:     "ORDERS",
    Subjects: []string{"orders.*"},
})

// Publish
js.Publish("orders.new", []byte(`{"order_id":"abc123"}`))

// Durable consumer
sub, _ := js.PullSubscribe("orders.*", "processor")
msgs, _ := sub.Fetch(10)
for _, msg := range msgs {
    msg.Ack()
}

Performance

  • Throughput: 10M+ messages/second
  • Latency: Sub-millisecond publish latency
  • Connections: 100K+ concurrent connections per node

Troubleshooting

IssueCauseSolution
JetStream not enabledMissing flagAdd --jetstream to command
Cluster routes not connectingAuth mismatchVerify ruser:T0pS3cr3t matches all nodes
Kafka clients can't connectRedpanda not runningStart Kafka sidecar container
Data lost on restartNo persistent volumeMount volume at --store_dir path
  • hanzo/hanzo-storage.md - S3-compatible object storage (uses PubSub for event notifications)
  • hanzo/hanzo-platform.md - PaaS deployment
  • hanzo/hanzo-universe.md - Production K8s infrastructure
  • hanzo/hanzo-database.md - PostgreSQL database

How is this guide?

Last updated on

On this page