Method v1.1 · Rules v0.52, Part N

How A1 works

Whether you can run accountable agents on an AI product: see what it costs and how much it is used, tie both to one agent, control spend and access, and have agents report to an outside register. It does not measure how good the AI is.

A1 is written into the Rules as Part N: twenty-six numbered clauses, versioned like every other Part. This page explains them; the clauses govern. The formula is reproducible from the data, and every day's ratings are sealed into the transparency log.

PillarWeightWhat scores well
Can you see what it costs?25%A cost API you can call scores highest; an admin console or cloud billing export next; a card statement or emailed invoice is all some products leave you. Published prices add to it.
Can you see how much it's used?15%A usage API (tokens, seats, credits, conversations) a buyer can read.
Can you tie cost to one agent?15%Exact (one key or project per agent) beats by-project, by-workspace or by-user.
Can an agent on it report out?15%A remote MCP connector scores highest, then a local MCP server, an SDK, an API or a workflow step; copy-and-paste lowest. Any open standard counts, not just FleetRegistry.
Can you control spend and access?20%Hard spend caps (30%), scoped per-agent keys (25%), an audit log you can read by API (20%), revoking by API (15%), SSO with SCIM (10%).
Does it work on small plans?10%Whether the plan a person or small business buys gets the cost and usage data at all.

Fleets first

There are two kinds of evidence. Desk research is what we read in a vendor's own pricing pages, API references and admin docs. Fleet evidence is what registered agents actually show: a cost report that came through, usage pulled by API, a key per agent, an agent that checked in. Fleet evidence comes only from independent accounts; FleetRegistry's own fleets and test sign-ups never count.

An independent fleet is an account at least fourteen days old, not ours or our staff's or test accounts, with an agent that reported on two different days; one company domain counts once (Rule N3.1). Each independent fleet counts as one observation. Once three fleets have reported on a pillar, desk research counts for at most half as much as the fleets, so the people running agents on a product decide at least two-thirds of that pillar.

A neutral start, and ranges

Every fact carries its own rung of the register's evidence ladder: Verified when the weekly independent re-check confirms it, Observed when an independent fleet shows it, Declared when the vendor's own material says so, Inferred when read from indirect material. Every pillar starts at a neutral 50 worth one fact, so a product never looks perfect or hopeless on one thin source. Facts count by rung: verified 6, observed 4, declared 4, 2.5 or 1.5 by confidence, inferred 1; the weakest fact in a pillar sets its weight. The ± on every score is a 95% range from how much evidence there is and how good it is. A pillar we know nothing about sits at 50 with a wide range. A rank gets "≈" when the range is wider than ±11.

A1 needs proof

A score of 85 or more only earns A1 when at least one independent fleet shows the product's cost data actually flows, and no known pillar is below 40. Until then it shows as A, marked A1 on proof. Paper alone never makes A1.

Living ratings

Ratings are re-computed continuously and saved daily, so every product has a history. A desk fact keeps full weight for 180 days, then halves every 180 days until it is checked again. When a vendor ships a billing API or a connector, the rating moves; a move of 15 points or more shows on the front page.

Rubrics by kind of product

A framework owes you different things than a chat app. The weights change with the kind of product (Rule N7.1):

KindCostUsagePer agentReportsControlsSmall plans
Agents and agent platforms
The default: what it costs and who controls it matter most.
251515152010
Assistants (chat apps people buy seats for)
Seat plans rarely split cost per agent; whether an agent on it can report out, and whether the small plan gets anything, matter more.
251510201515
Devices
A device is bought outright or by subscription; what that costs, and who can switch it off, matter most.
301510152010
Features inside a larger product
Cost is often bundled into the host product, so usage visibility carries more.
202015152010
Open-source frameworks
The bill comes from the model you plug in; what the framework owes you is telemetry, per-agent attribution and a way to report out.
152020251010
Platforms and APIs
The default.
251515152010

No downgrade without notice

From 9 October, when the evidence puts a product a grade or more lower, a notice is published beside the rating and the published grade stands for seven days. The vendor, or anyone, may reply with a source; a reply holds the grade until a named reviewer decides and publishes the reason (Rules N4.1 to N4.2). Upgrades take effect at once.

We measure our own research

From 5 October, every week a run that did not record the facts re-checks one in ten unverified facts, without seeing what was recorded. The register compares the two. Facts that hold become verified; facts that don't become corrections. Our accuracy over the last 28 days, with its range, is published on the front page, good or bad (Rule N6).

What isn't graded

Vendors

A vendor's rating averages its graded products, each weighted by 1 + ln(1 + fleets running it), and pulled toward 50 by one product's worth. Vendors are ranked once two products are graded.

Track record and incidents

When a vendor announces something with a date (a usage API, a connector, spend caps), it is logged with its source and settled as a hit, partial or miss when the date passes. Incidents (billing surprises, unannounced price changes, admin data lost) are hand-written, need two sources and include the vendor's response. Both are shown beside the rating and never change the score.

Our conflict, disclosed

FleetRegistry runs a register for AI agents. "Can an agent report out" credits any open standard (MCP, an SDK, an API, webhooks), never FleetRegistry specifically, and FleetRegistry's own fleets never count as evidence.

Corrections

Anyone, vendors included, can send a correction with a source. An accepted correction supersedes the fact, is published in the corrections register with an FR-C number, and moves the rating at the next run. A fact corrected at a vendor's request does not move again at that vendor's request for thirty days (Rule N5).

Next

As the register grows: whether an agent reports its true model, whether its account of itself holds up, and how far its actual cost sits from the price list.

See the ratings

1134 of 1313 live AI products rated so far.