# List compare

URL: https://docs.hanzo.ai/docs/openapi/benchmark/get-benchmark-compare

> The ONLY valid arm-vs-arm test: it pairs the two models on the items BOTH completed, and answers rescue and damage counts with an exact-McNemar p.

`GET /v1/benchmark/compare`

> [Benchmark →](https://docs.hanzo.ai/docs/openapi/benchmark) · [All API references →](https://docs.hanzo.ai/docs/openapi) · [Get started →](https://docs.hanzo.ai/docs/guides)

|               |                                             |
| ------------- | ------------------------------------------- |
| **Address**   | `https://api.hanzo.ai/v1/benchmark/compare` |
| **Method**    | `GET`                                       |
| **Operation** | `get_benchmark_compare`                     |
| **Auth**      | `Authorization: Bearer $HANZO_API_KEY`      |

The ONLY valid arm-vs-arm test: it pairs the two models on the items
BOTH completed, and answers rescue and damage counts with an exact-McNemar p.

Pairing is what prevents the subset artifact — comparing one model's easy subset
against another's full run — so n\_common, not either arm's own coverage, is the
number to read this by.

Both a and b are required. The benchmark defaults to gpqa\_diamond.

## Request

3 fields.

| Field       | In    | Type   | Required | Description                                                             |
| ----------- | ----- | ------ | -------- | ----------------------------------------------------------------------- |
| `benchmark` | query | string | —        | Benchmark is the catalog id to compare on, defaulting to gpqa\_diamond. |
| `a`         | query | string | yes      | A is the first model id.                                                |
| `b`         | query | string | yes      | B is the second model id.                                               |

## Response

| Status    | Body              | Meaning |
| --------- | ----------------- | ------- |
| `200`     | benchmark.pairing | ok      |
| `default` | problem-details   | refused |

`200` body — 10 fields.

| Field             | In   | Type            | Always | Description                                                              |
| ----------------- | ---- | --------------- | ------ | ------------------------------------------------------------------------ |
| `a`               | body | string          | —      | A is the first model id.                                                 |
| `a_correct`       | body | integer (int64) | —      | ACorrect is how many of those common items A got right.                  |
| `b`               | body | string          | —      | B is the second model id.                                                |
| `b_correct`       | body | integer (int64) | —      | BCorrect is how many of those common items B got right.                  |
| `benchmark`       | body | string          | —      | Benchmark is the catalog id the two arms were compared on.               |
| `mcnemar_p`       | body | number (double) | —      | McnemarP is the two-sided exact binomial p on the discordant pairs.      |
| `n_common`        | body | integer (int64) | —      | NCommon is how many items BOTH arms completed.                           |
| `net_a_minus_b`   | body | integer (int64) | —      | NetAMinusB is the two rescue counts subtracted — A's advantage in items. |
| `rescue_a_over_b` | body | integer (int64) | —      | RescueAOverB is how many items A got right and B got wrong.              |
| `rescue_b_over_a` | body | integer (int64) | —      | RescueBOverA is how many items B got right and A got wrong.              |

Failure carries the platform error shape — see [Errors](https://docs.hanzo.ai/docs/errors).

## Examples

**CLI**

```bash
hanzo benchmark compare \
  --a <a> \
  --b <b>
```

**SDK**

**.ts**

```ts
import { Configuration, BenchmarkApi } from 'hanzoai';

const api = new BenchmarkApi(new Configuration({ accessToken: process.env.HANZO_API_KEY }));
const { data } = await api.getBenchmarkCompare({ a: 'a', b: 'b' });
```

**.py**

```python
from hanzoai.cloud import ApiClient, Configuration
from hanzoai.cloud.api import BenchmarkApi

client = ApiClient(Configuration(access_token=os.environ["HANZO_API_KEY"]))
result = BenchmarkApi(client).get_benchmark_compare(a='a', b='b')
```

**.go**

```go
cfg := hanzoai.NewConfiguration()
cfg.AddDefaultHeader("Authorization", "Bearer "+os.Getenv("HANZO_API_KEY"))
client := hanzoai.NewAPIClient(cfg)

resp, _, err := client.BenchmarkAPI.GetBenchmarkCompare(context.Background()).Execute()
if err != nil {
	return err
}
```

**.rs**

```rust
use hanzo_client::apis::{configuration::Configuration, benchmark_api};

let mut cfg = Configuration::new();
cfg.bearer_access_token = std::env::var("HANZO_API_KEY").ok();

let result = benchmark_api::get_benchmark_compare(&cfg, Default::default()).await?;
```

**.java**

```java
import ai.hanzo.cloud.ApiClient;
import ai.hanzo.cloud.api.BenchmarkApi;

ApiClient client = new ApiClient();
client.setBearerToken(System.getenv("HANZO_API_KEY"));

var result = new BenchmarkApi(client).getBenchmarkCompare();
```

**HTTP**

```bash
curl https://api.hanzo.ai/v1/benchmark/compare?a=%3Ca%3E&b=%3Cb%3E \
  -H "Authorization: Bearer $HANZO_API_KEY"
```

**MCP**

Tool `benchmark`, op `get_benchmark_compare` — POST the JSON-RPC envelope to `https://api.hanzo.ai/v1/mcp`.

```bash
curl -X POST https://api.hanzo.ai/v1/mcp \
  -H "Authorization: Bearer $HANZO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
       "jsonrpc": "2.0",
       "id": 1,
       "method": "tools/call",
       "params": {
         "name": "benchmark",
         "arguments": {
           "op": "get_benchmark_compare",
           "input": {
             "a": "<a>",
             "b": "<b>"
           }
         }
       }
     }'
```

---

[Benchmark API](https://docs.hanzo.ai/docs/openapi/benchmark) · [All Hanzo APIs](https://docs.hanzo.ai/docs/openapi) · [Interactive reference](https://docs.hanzo.ai/docs/openapi/reference)