Annotation Queues

An annotation queue is a named work list of traces and observations for humans to review and score. Each queue is bound to a set of score configs, so every reviewer applies the same rubric. Served by Hanzo O11y, tenant-scoped by org.

How it works

1. A queue declares which score configs reviewers must fill in (scoreConfigIds).
2. Traces or observations are added to the queue as items, each with a status and optional assignee.
3. A reviewer opens an item, applies the rubric, and the result is written back as a score with source: ANNOTATION.

Related

• Scores — what a completed review produces
• Score Configs — the rubric a queue enforces
• Traces — the items reviewers score