Hanzo Storage

Overview

Hanzo Storage is a high-performance, S3-compatible object storage server for AI workloads. This is a full server binary, not a client library: you run it, point any S3 SDK at it, and it answers the S3 API. aws-sdk-go, boto3, @aws-sdk/client-s3 and the aws CLI all work unchanged.

Hanzo Storage is the platform's object substrate. Model artifacts, training datasets, Datastore's columnar parts, and encrypted per-project database files all land here, which is why it is tuned for large objects and many concurrent readers rather than for tiny-object churn.

Quick reference

Quick start

Docker

docker run -p 9000:9000 -v hanzo-s3:/data \
  ghcr.io/hanzoai/s3:latest server -s3 -s3.port=9000 -dir=/data

The image's entrypoint is the s3 binary, so anything after the image name is passed straight through as flags.

Build from source

make install                       # builds and installs the `s3` binary
s3 server -s3 -s3.port=9000 -dir=/data

server is the all-in-one mode: it starts the master, a volume server, the filer, and the S3 gateway in one process. Add -filer, -volume.max, -master.volumeSizeLimitMB and friends to shape a single-node dev cluster.

Credentials

There is no default account. Access keys come from an identities file passed with -s3.config:

{
  "identities": [
    { "name": "anonymous", "actions": ["Read"] },
    {
      "name": "app",
      "credentials": [
        { "accessKey": "…", "secretKey": "…" }
      ],
      "actions": ["Read", "List", "Write", "Tagging"]
    }
  ]
}

s3 server -s3 -s3.config=./s3.json -dir=/data

Per-identity actions are the authorization model: Admin, Read, List, Write, Tagging, and the fine-grained Read:<bucket> / Write:<bucket> forms. Keep the file in Hanzo KMS and mount it at runtime — never bake keys into an image.

Verify connectivity

export AWS_ACCESS_KEY_ID=…  AWS_SECRET_ACCESS_KEY=…
aws --endpoint-url http://localhost:9000 s3 mb s3://my-bucket
aws --endpoint-url http://localhost:9000 s3 cp ./model.safetensors s3://my-bucket/
aws --endpoint-url http://localhost:9000 s3 ls s3://my-bucket/

Admin console

s3 admin -port=23646 -master=localhost:9333

Key features

• Full S3 API surface — buckets, objects, multipart, presigned URLs, tagging, CORS, object lock and retention
• Replication placement per volume (-defaultReplication), plus erasure coding for cold data
• Server-side encryption: SSE-C, SSE-S3 and SSE-KMS, with KMS backed by Hanzo KMS or Lux
• Bucket policies, versioning, and lifecycle rules (expiration, transition)
• Multi-tenancy through per-identity credentials and bucket-scoped actions
• Event notifications to Hanzo PubSub, Kafka, AMQP, MQTT and webhooks
• Prometheus metrics on :9324
• Admin console for topology, volume and bucket management
• Filer with FUSE mount, so a bucket can be a POSIX path when something needs one

Client SDKs

Any S3-compatible SDK works — that is the point of the interface:

Build and test

make install        # build + install the `s3` binary
make server         # run a local all-in-one dev server
make test           # unit tests
make benchmark      # throughput benchmark