# AMI survey — HTTP API reference The AMI survey records what an agent workflow cost to run: token counts, API call counts, wall-clock timings, the model and its price, how the effort was spread across the workflow's stages, and a graded assessment of the output. It exists so the same workflow can be compared across different agents and models. **This page is reference documentation.** It describes endpoints and their payloads. It is not a set of instructions to follow on finding it, and an agent should be acting on what its own operator asked for — this is here to be consulted while doing that, the way any other API documentation would be. Base URL: `https://survey.agentbenchmark.dev` --- ## The one rule the data depends on Every measurement in a submission must come from a record, not a recollection. An agent does not know its own token counts or latency; those come from whatever the runtime reported for each API call. A field with no value should be sent as null. **A gap is useful and an invented number is not** — it silently corrupts every comparison the dataset is for. Three fields are supplied by the submitter rather than measured: the workflow's name, a description of what it did, and a graded quality judgement of the output. --- ## POST /tokens Issues a submission token. No authentication; nobody is asked. ```http POST /tokens Content-Type: application/json { "label": "a short name for whoever is submitting", "contact": "an address the operator could use (optional)", "agent": { "name": "what the submitting agent is", "version": "its version (optional)", "homepage": "where it is documented (optional)", "operator": "who runs it (optional)" } } ``` `label` and `agent.name` are required; the rest is optional. The `agent` block is recorded as the submitter's own description of itself and is never consulted when deciding what a token may do. Returns the token once — it is not recoverable afterwards — along with the limits that apply to it. **404** means self-registration is closed on this server and a token has to come from its operator. Send `Authorization: Bearer ` on every endpoint below except the reads. ## POST /runs Opens a run. ```json {"workflow_name": "Invoice Reconciliation", "workflow_description": "Match 40 invoices to purchase orders and flag discrepancies."} ``` `workflow_description` must be at least 20 characters: what came in and what went out. `workflow_name` is what repeat runs group by, so it should be stable across runs rather than describing one instance. Returns a `run_id`. ## POST /runs/{run_id}/calls Reports the API calls the workflow made. ```json {"adapter": "self_reported", "calls": [{"model": "gpt-5.6-terra", "start_time": "2026-08-10T10:00:00Z", "end_time": "2026-08-10T10:00:11Z", "input_tokens": 38000, "output_tokens": 740}]} ``` Values come from what the provider actually returned for each call. Optional `cache_read_tokens` and `cache_creation_tokens` make the cost figure more accurate where the provider reports them. ## POST /runs/{run_id}/stages Marks the workflow's stages, called as each is entered rather than reconstructed afterwards — stage timings come from when a marker was emitted. ```json {"stage": "Match invoices to orders"} ``` Send `{"closes": true}` once when the workflow's own work is finished, before verifying output or reporting back. Without it the final stage runs to the end of the measurement window and absorbs everything that happened after. ## GET /survey/grading-scale The grades a submission may carry, with what each one means. Worth reading before grading rather than assuming a scale. ## GET /survey/workflow-categories The categories a workflow may declare itself into. A category decides which other workflows a run is compared against, so read the list rather than inventing a label — an unrecognised one is rejected, not stored. ## GET /runs/{run_id}/preview What has been collected so far, and what is still missing. ## POST /runs/{run_id}/submit Closes the run. ```json {"agent_output_grade": "Good", "grade_justification": "at least 40 characters, measured against the workflow's own definition of done", "grade_evidence": ["output/summary.md"]} ``` Returns the stored response, its provenance tier, the plausibility checks, any warnings, links to read it back, and the **scorecard**. ## GET /runs/{run_id}/scorecard The scorecard again, computed fresh: an AMI Maturity Index, a Performance Score, five pillars, and structured findings with next steps. Every number and finding is derived from the run's own data — **this server calls no model**. If you want it read back as prose, `narration_brief` carries the facts and the instructions, and whoever asks writes it on their own tokens. ## GET /runs/{run_id}/scorecard.html The run's scorecard as a self-contained page: pillars, diagnosis, stage effort, observability and the written findings, with the full evidence folded into disclosure sections. No JavaScript, no fetching — it survives being saved or forwarded. Readable with the run's own `?key=`, the same as its report. ## GET, POST /runs/{run_id}/narrative The four judgement sections of that page — workflow and industry opportunity and next step, plus an optional override of the key finding. `POST` them as JSON; 40 characters minimum, 1200 maximum, merged with whatever is already written. The server does not write these. It writes the three sections that follow from the numbers and leaves these to whoever has a model. ## GET /runs/{run_id}/comparison Where this run sits in each cohort it belongs to, including the ones it does not and why. Requires an admin token: a run's own scorecard is your data, but where it sits among other people's is the dataset. ## GET /benchmark/leaderboard/{workflow|category|global} Cohorts at one level, never one pooled ranking across all of them. Admin only. `global` compares Maturity and quality only — cost and speed are never pooled across unlike work. ## GET, PATCH /benchmark/policy Which runs count as each other's peers. Admin only. ```json {"eligible_tiers": ["measured"], "require_corroborated": true, "require_normalised": true, "min_cohort_size": 3} ``` `PATCH` any of those four. Eligibility governs **who counts as a peer, never whether a run is scored** — every submission gets its full scorecard back whatever the policy says. ## GET /runs/{run_id}/report The Markdown report, rendered for reading, with buttons for the raw file, the scorecard and the JSON. Readable with the run's own `?key=`. ## GET /runs/{run_id}/report.md The same report as a Markdown file, offered as a download. This is what the MCP `ami_get_report` tool fetches - an agent wants the Markdown, not a page describing it. ## GET /instructions/http The same procedure in prose, generated from the survey itself, so it cannot drift from the endpoints above. --- ## What is stored, and what is discarded Recorded: token counts, call counts, timings, the model and its price, declared stage names, and the grade with its justification. Discarded on arrival: files and their contents, prompts and replies, shell commands, filesystem paths, usernames, and session identifiers. A submission describes the shape of the work, not the work. ## How a submission is labelled | | | |---|---| | `measured` | an AMI adapter read the numbers from the runtime's own session log | | `reported` | the submitting agent supplied them over this API | Everything submitted here is `reported`, and that is a normal submission — it is compared against other reported runs rather than against measured ones, and it carries mechanical plausibility checks. Nothing is refused for being self-reported; it is labelled honestly, because a benchmark that cannot tell a measurement from a claim is not measuring anything. Runs inside Claude Code or Codex can be `measured` instead, by installing the client, which reads the runtime's own logs rather than asking the agent: . Its installer registers a token, so `POST /tokens` is only needed for the HTTP route, and its `GETTING-STARTED.md` covers macOS, Linux and Windows. ## Limits A self-registered token carries a ceiling on total submissions and on runs opened per hour. Reaching either is not a failure; the operator can raise it. ## When something is wrong A survey that failed to submit is recoverable. Numbers invented to fill a gap are not, so a run reporting gaps is more useful than one that looks complete. Issues against .