How A1 works
Whether you can run accountable agents on an AI product: see what it costs and how much it is used, tie both to one agent, control spend and access, and have agents report to an outside register. It does not measure how good the AI is.
A1 is written into the Rules as Part N: twenty-six numbered clauses, versioned like every other Part. This page explains them; the clauses govern. The formula is reproducible from the data, and every day's ratings are sealed into the transparency log.
Fleets first
There are two kinds of evidence. Desk research is what we read in a vendor's own pricing pages, API references and admin docs. Fleet evidence is what registered agents actually show: a cost report that came through, usage pulled by API, a key per agent, an agent that checked in. Fleet evidence comes only from independent accounts; FleetRegistry's own fleets and test sign-ups never count.
An independent fleet is an account at least fourteen days old, not ours or our staff's or test accounts, with an agent that reported on two different days; one company domain counts once (Rule N3.1). Each independent fleet counts as one observation. Once three fleets have reported on a pillar, desk research counts for at most half as much as the fleets, so the people running agents on a product decide at least two-thirds of that pillar.
A neutral start, and ranges
Every fact carries its own rung of the register's evidence ladder: Verified when the weekly independent re-check confirms it, Observed when an independent fleet shows it, Declared when the vendor's own material says so, Inferred when read from indirect material. Every pillar starts at a neutral 50 worth one fact, so a product never looks perfect or hopeless on one thin source. Facts count by rung: verified 6, observed 4, declared 4, 2.5 or 1.5 by confidence, inferred 1; the weakest fact in a pillar sets its weight. The ± on every score is a 95% range from how much evidence there is and how good it is. A pillar we know nothing about sits at 50 with a wide range. A rank gets "≈" when the range is wider than ±11.
A1 needs proof
A score of 85 or more only earns A1 when at least one independent fleet shows the product's cost data actually flows, and no known pillar is below 40. Until then it shows as A, marked A1 on proof. Paper alone never makes A1.
Living ratings
Ratings are re-computed continuously and saved daily, so every product has a history. A desk fact keeps full weight for 180 days, then halves every 180 days until it is checked again. When a vendor ships a billing API or a connector, the rating moves; a move of 15 points or more shows on the front page.
Rubrics by kind of product
A framework owes you different things than a chat app. The weights change with the kind of product (Rule N7.1):
No downgrade without notice
From 9 October, when the evidence puts a product a grade or more lower, a notice is published beside the rating and the published grade stands for seven days. The vendor, or anyone, may reply with a source; a reply holds the grade until a named reviewer decides and publishes the reason (Rules N4.1 to N4.2). Upgrades take effect at once.
We measure our own research
From 5 October, every week a run that did not record the facts re-checks one in ten unverified facts, without seeing what was recorded. The register compares the two. Facts that hold become verified; facts that don't become corrections. Our accuracy over the last 28 days, with its range, is published on the front page, good or bad (Rule N6).
What isn't graded
- Early access: products in beta or only announced show their facts but get no grade, so early numbers never follow them into release.
- Not sold: internal or bundled tools nobody buys on their own.
- Thin: a grade needs at least 60% of the method's weight measured.
Vendors
A vendor's rating averages its graded products, each weighted by 1 + ln(1 + fleets running it), and pulled toward 50 by one product's worth. Vendors are ranked once two products are graded.
Track record and incidents
When a vendor announces something with a date (a usage API, a connector, spend caps), it is logged with its source and settled as a hit, partial or miss when the date passes. Incidents (billing surprises, unannounced price changes, admin data lost) are hand-written, need two sources and include the vendor's response. Both are shown beside the rating and never change the score.
Our conflict, disclosed
FleetRegistry runs a register for AI agents. "Can an agent report out" credits any open standard (MCP, an SDK, an API, webhooks), never FleetRegistry specifically, and FleetRegistry's own fleets never count as evidence.
Corrections
Anyone, vendors included, can send a correction with a source. An accepted correction supersedes the fact, is published in the corrections register with an FR-C number, and moves the rating at the next run. A fact corrected at a vendor's request does not move again at that vendor's request for thirty days (Rule N5).
Next
As the register grows: whether an agent reports its true model, whether its account of itself holds up, and how far its actual cost sits from the price list.
See the ratings
1134 of 1313 live AI products rated so far.