Is per-variant conversion, lift and statistical significance against the…

POST /v1/experiment/{id}/analyze

Is per-variant conversion, lift and statistical significance against the control arm.

It reads per-subject outcomes from the analytics plane over a window, folds them into per-variant samples, and returns each arm's exposed count, conversions, rate, lift versus control, two-proportion z, two-tailed p-value and whether it clears alpha. Arms with no data still appear with zero exposed, so the read is complete over the experiment's declared arms; the control arm sorts first. The pooled-variance estimator is used and the p-value is exact; a degenerate comparison (an empty arm, no variance) answers z 0 and p 1 — not significant, never an error.

Only EXPOSED subjects are counted, and each is joined to its arm by re-evaluating the assignment flag AT ANALYSIS TIME — not from what was in force during the window. That is the one rule to get right: analyzing an experiment after its winner has been promoted re-buckets every subject into the promoted arm, collapsing the control to zero exposed and making the result meaningless. Read the analysis before deciding. A subject the flag cannot place is dropped rather than allowed to poison the fold.

The winner in the response is ADVISORY — the significant, control-beating arm with the highest rate, or empty when inconclusive. It promotes nothing; the decision is a separate, explicit act.

Every plane read is scoped to the caller's org. Per-variant samples are also written to the research evidence plane as immutable ab rows, best-effort: the analysis is still returned if that write fails, because the samples are recomputable, and the failure is logged rather than swallowed.

Request

6 fields, body application/json (required).

Response

200 body — 15 fields.

Failure carries the platform error shape — see Errors.

Examples

hanzo experiments analyze <id>

import { Configuration, ExperimentApi } from 'hanzoai';

const api = new ExperimentApi(new Configuration({ accessToken: process.env.HANZO_API_KEY }));
const { data } = await api.postExperimentByIdAnalyze({ id: 'id', alpha: 0, days: 0 });

from hanzoai.cloud import ApiClient, Configuration
from hanzoai.cloud.api import ExperimentApi

client = ApiClient(Configuration(access_token=os.environ["HANZO_API_KEY"]))
result = ExperimentApi(client).post_experiment_by_id_analyze(id='id', alpha=0, days=0)

cfg := cloud.NewConfiguration()
cfg.AddDefaultHeader("Authorization", "Bearer "+os.Getenv("HANZO_API_KEY"))
client := cloud.NewAPIClient(cfg)

resp, _, err := client.ExperimentAPI.PostExperimentByIdAnalyze(context.Background()).Execute()
if err != nil {
	return err
}

use hanzo_cloud::apis::{configuration::Configuration, experiment_api};

let mut cfg = Configuration::new();
cfg.bearer_access_token = std::env::var("HANZO_API_KEY").ok();

let result = experiment_api::post_experiment_by_id_analyze(&cfg, Default::default()).await?;

import ai.hanzo.cloud.ApiClient;
import ai.hanzo.cloud.api.ExperimentApi;

ApiClient client = new ApiClient();
client.setRequestInterceptor(b -> b.header("Authorization", "Bearer " + System.getenv("HANZO_API_KEY")));

var result = new ExperimentApi(client).postExperimentByIdAnalyze();

curl -X POST https://api.hanzo.ai/v1/experiment/<id>/analyze \
  -H "Authorization: Bearer $HANZO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
       "alpha": 0,
       "days": 0
     }'

curl -X POST https://api.hanzo.ai/v1/mcp \
  -H "Content-Type: application/json" \
  -d '{
       "jsonrpc": "2.0",
       "id": 1,
       "method": "tools/call",
       "params": {
         "name": "describe",
         "arguments": {
           "op": "list_experiments"
         }
       }
     }'

Experiment API · All Hanzo APIs · Interactive reference