We Re-Ran the Study on 147 Brands. Two of Our Numbers Were Wrong.

A 58-domain sample is small enough that any striking result deserves a second look before anyone builds on it. So we widened the frame to 147 brands, turned the collection into a script that anyone can re-run, and checked whether what we published still holds.

Mostly it does. Seventy of 147 brands serve a live agent-commerce discovery document — 48% of the whole list and 65% of the ones that answered us at all. The headline survives 2.5× the sample.

Two things did not survive, and they are the reason this post exists.

Correction: some brands do block answer engines

We previously reported that not one brand in our sample disallowed AI answer engines in robots.txt. On 147 domains, two do: catbird.com and ourplace.com. Both shut out the full set — the search-facing crawlers, the training crawlers and the live agents fetching on behalf of a user — which reads like a deliberate policy rather than a copied snippet.

Six brands disallow training crawlers, and the variation between them is the interesting part. Two block everything in that class. Warby Parker blocks only CCBot and Bytespider, leaving GPTBot and Google-Extended free. Chomps blocks GPTBot, CCBot and Google-Extended but not ClaudeBot. These are choices, not defaults.

So the accurate claim is not “nobody blocks”. It is that blocking remains rare and highly selective: 2 of 147 for answer engines, 6 of 147 for training. The direction of the earlier finding stands; the absolute has moved off zero, and anyone who quoted our zero should quote this instead.

Correction: refusal at the edge is far more common than we found

The first sample had seven of 30 domains refusing or failing to answer. At 147, it is 39 — just over a quarter.

The limitation that came with the first study applies here with more weight, so it is worth restating plainly: our client identifies itself honestly and no bot vendor recognises the name. A 403 proves the site refuses unfamiliar crawlers. It does not prove the site refuses OAI-SearchBot, which arrives with a name operators know and IP ranges they can verify. Twenty-seven per cent is an upper bound on edge blocking, not a measurement of it.

It is still the number worth knowing, because it is invisible to every audit that reads robots.txt. Nearly all of these 39 domains publish a permissive robots.txt. The file says yes and the edge says no, and only one of those two is ever checked.

What held, and what the wider frame added

The expansion deliberately reached for categories the first list was thin on and for brands large enough to run their own infrastructure. That pulls the rate down exactly as the platform-default explanation predicts:

Every one of the 70 documents reports the same protocol version. Sixty-nine of them answer a tool-listing request with the same thirteen tools. That uniformity is the finding: this is one vendor’s rollout observed 70 times, not 70 independent integrations, and it is why a brand can have this switched on without anyone internally having decided to.

Two brands that break the pattern

diff.com serves a valid discovery document and then answers the tool-listing request with nothing at all — the only one of 70 to do so. It is also one of three registering no Shop Pay handler. Something is half-configured there, and a merchant would have no way of knowing.

olukai.com is the mirror image. It publishes the generated llms.txt that names the agent-commerce endpoints, and the endpoint it names returns nothing. The file advertises a capability the domain does not currently serve.

Both matter more than curiosities usually do, because they are the cases a merchant cannot detect from inside. Nothing errors. Nothing appears in a dashboard.

The decoupling held too

Twelve of the 70 brands accepting an agent checkout publish an llms.txt that never mentions it, and seven of those publish no such file at all. Across the whole list: 59 serve the generated template, 18 wrote their own, 58 serve none, and 12 we could not determine because the domain refused us.

Which keeps the practical conclusion from the earlier study intact at a larger scale — the file is a poor proxy for the capability, in both directions.

A gap we expected to find, and did not

Being able to transact with a store through a protocol and being able to read its pages are different capabilities, and the first arrived as a platform default while the second requires markup somebody has to emit. We assumed that would show up as a gap, and went looking for it: for each of the 70 brands with a live discovery document, we pulled a real product page from the sitemap and read it with no JavaScript.

There is no gap. Of the 68 where a product page could be sampled, 64 carry product structured data and all 68 expose a price in the raw HTML. Four have no product markup: brooklinen.com, gymshark.com, alohas.com and quay.com.

That is worth reporting precisely because it is dull. The stores an agent can buy from are, almost without exception, stores an agent can also read — and both properties came from the same place. Adobe measured US retail product pages at 66% machine readability across the sector; this population sits far above that, and the difference is not effort. It is which platform the brand is on.

How we nearly published the opposite

The first version of that measurement said 30% of agent-transactable stores had no product markup. It was wrong three times over, and the errors are worth naming because each one is easy to make and none of them throws.

• It matched only @type: Product. Several brands emit ProductGroup, which is correct markup for a product with variants — Allbirds and Skims were both counted as having none.
• Its product-URL filter matched nested sitemap files. Two brands were “scored” on an XML document.
• It took the first matching URL from the sitemap, which for two brands was a gift card.

Each bug pushed the number in the same direction, toward a more interesting result. That is the pattern to be suspicious of. We caught it by spot-checking four brands against the live pages before writing anything — which is the entire reason the corrected sampler exists, and the reason this section is here rather than a headline about a third of stores being unreadable.

Re-run it yourself

The collection is a script now, and that is the substantive change from the first study. The earlier numbers were gathered by hand, which meant nobody — including us — could reproduce them. Four read-only requests per domain, no carts, no orders:

curl -s https://kosas.com/.well-known/ucp | jq '.ucp.version'

curl -s -X POST https://kosas.com/api/ucp/mcp \
  -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' | jq '.result.tools | length'

Content type is checked as well as status code, which is not fussiness: one brand returns HTTP 200 at /llms.txt and serves two megabytes of HTML. Any survey counting status codes records that as adoption, and every published adoption figure derived that way is inflated by an unknown amount.

The sample frame lives in the repository as a plain list. It is hand-assembled for recognisability among US direct-to-consumer brands — not random, not exhaustive, and skewed toward companies that grew up online, which is the population most likely to have both modern crawler policy and platform-shipped agent endpoints. Eleven candidate domains were removed before collection because they did not resolve in DNS; leaving our own typos in the frame would have inflated the refusal count.

Per-brand results are in the DTC AI crawler index, free to cite with attribution. The original study is here, and this page explains how to run the same two requests against your own store.