On September 1, 2026 we ran the Shelfgrade scanner across 40 well-known DTC storefronts — the kind of brands with real SEO teams and seven-figure ad budgets. Each scan evaluated robots.txt against the crawlers that gate AI-shopping visibility (per RFC 9309), and checked Product/Offer JSON-LD on product pages sampled from each store’s own navigation. All 40 stores were reachable; none errored.
The headline: these are well-run stores, and the average readiness score was 90/100 — but the failures cluster in exactly the places AI surfaces read.
- 8 of 40 (20%) block PerplexityBot outright. Perplexity’s own docs recommend allowing it for your site to appear in results. These stores have opted out of an entire answer surface — almost certainly via a blanket “block the AI bots” robots.txt recipe that conflates training crawlers with search crawlers.
- 0 of 40 block OAI-SearchBot. The ChatGPT gate is intact everywhere — which makes the Perplexity number stranger: the same recipes that spare OpenAI’s search bot catch Perplexity’s.
- 8 of 40 (20%) ship no Product JSON-LD at all on their sampled product pages — nothing for Google’s merchant-listing pipeline (which feeds AI Overviews and the Shopping Graph behind Gemini) or Perplexity’s crawl-fed index to read.
- 20 of 40 (50%) are missing GTIN in product markup, the identifier Google-compatible feed parsing can require, and 23 of 40 (57%) omit
shippingDetailswhile 22 of 40 (55%) omit return-policy markup — the Offer-level fields that answer “how much, shipped, and can I return it” without a click.
Grade distribution: A=30 · B=1 · C=9. No store failed outright — and that is the point. The gap between a 90-average cohort and full AI-surface readiness is a handful of concrete, fixable omissions, not a rebuild. The stores losing Perplexity visibility are one robots.txt line away from getting it back.
Methodology notes, for the skeptical (correctly so): scans are domain-level — robots.txt evaluated per user-agent token against the vendors’ published crawler list, markup read from up to three product pages discovered from each store’s homepage or sitemap. A store’s feed (the other half of AI-shopping readiness) is not visible from the outside; these numbers understate total gaps. Store-level results aren’t published here — run the same scan on any store, including yours, free from the homepage.