Fetch one URL and read it back as markdown

POST /v1/crawl

Reads one URL and answers with the page as markdown.

It fetches a single URL from inside the cluster and answers with the address it actually landed on, the document's title, its content rendered to MARKDOWN, and whatever the page said about itself. One URL per call: batching would make the answer a partial-failure envelope every caller then has to unpack.

A PAGE THAT COULD NOT BE FETCHED IS A NORMAL ANSWER, not a fault. An unreachable host, a refused address and a content type that is not a document all answer 200 with success:false and the reason in error, because the caller sent a well-formed ask and gets a well-formed answer. Non-2xx is reserved for a caller problem — 400 with the same body when there is no url, 401 for a bad key, 503 when the surface is unconfigured — so error handling can trust the status. Check success before reading data.

Admission is either a validated principal or the shared service key, presented as X-API-Key or a Bearer; neither is refused, and an unset key fails closed rather than opening the fetcher to the private network. Pages are archived under the scope of the VERIFIED principal and NEVER a scope named in the body, so a URL already read under that scope is answered from the archive without touching the network; a service caller has no org and its pages land in the shared corpus.

The URL is caller-supplied and dialled from INSIDE the cluster, which makes this a request-forgery primitive by construction. Only http and https are accepted, and every address actually dialled must be public unicast — loopback, link-local, private and multicast are refused. The check lives in the DIALER rather than on the hostname, because resolving a name to validate it and then letting the transport resolve it again is a gap DNS rebinding walks straight through; redirects re-enter the same dialer.

Request

1 field, body application/json (required).

Response

200 body — 7 fields.

Failure carries the platform error shape — see Errors.

Examples

hanzo crawl create

import { Configuration, CrawlApi } from 'hanzoai';

const api = new CrawlApi(new Configuration({ accessToken: process.env.HANZO_API_KEY }));
const { data } = await api.readPage({ url: "<url>" });

from hanzoai.cloud import ApiClient, Configuration
from hanzoai.cloud.api import CrawlApi

client = ApiClient(Configuration(access_token=os.environ["HANZO_API_KEY"]))
result = CrawlApi(client).read_page(url="<url>")

cfg := cloud.NewConfiguration()
cfg.AddDefaultHeader("Authorization", "Bearer "+os.Getenv("HANZO_API_KEY"))
client := cloud.NewAPIClient(cfg)

resp, _, err := client.CrawlAPI.ReadPage(context.Background()).Execute()
if err != nil {
	return err
}

use hanzo_cloud::apis::{configuration::Configuration, crawl_api};

let mut cfg = Configuration::new();
cfg.bearer_access_token = std::env::var("HANZO_API_KEY").ok();

let result = crawl_api::read_page(&cfg, Default::default()).await?;

import ai.hanzo.cloud.ApiClient;
import ai.hanzo.cloud.api.CrawlApi;

ApiClient client = new ApiClient();
client.setRequestInterceptor(b -> b.header("Authorization", "Bearer " + System.getenv("HANZO_API_KEY")));

var result = new CrawlApi(client).readPage();

curl -X POST https://api.hanzo.ai/v1/crawl \
  -H "Authorization: Bearer $HANZO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
       "url": "<url>"
     }'

curl -X POST https://api.hanzo.ai/v1/mcp \
  -H "Authorization: Bearer $HANZO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
       "jsonrpc": "2.0",
       "id": 1,
       "method": "tools/call",
       "params": {
         "name": "crawl",
         "arguments": {
           "op": "read_page",
           "input": {
             "url": "<url>"
           }
         }
       }
     }'

Crawl API · All Hanzo APIs · Interactive reference