How AGMM treats crawlers and AI agents
Named AI training crawlers are asked to pay a small per-page fee in USDC via x402; people, search engines and answer engines read free.
Free for everyone
- People, in any browser.
- Search engines: Googlebot, Bingbot, Applebot, DuckDuckBot, YandexBot, Baiduspider, Yahoo Slurp.
- AI answer engines and assistants fetching a page for a person: OAI-SearchBot, PerplexityBot, Claude-SearchBot, Amzn-SearchBot, DuckAssistBot, ChatGPT-User, Perplexity-User, Claude-User, Amzn-User, MistralAI-User, Meta user fetcher. We want to be cited.
- Link previews: facebookexternalhit, WhatsApp, Slackbot, LinkedInBot, Twitterbot, Discordbot, TelegramBot.
- The front door for every model: the home page, services, about, contact,
/llms.txt, sitemaps and/about/agmm.json.
A small fee per page
Named AI training collectors (meta-externalagent, GPTBot, ClaudeBot, CCBot, Bytespider, Amazonbot, cohere-ai, AI2Bot) are asked to pay a small per-page fee in USDC for long-form pages. The amount is stated in each machine-readable offer and in /x402/catalog.json. The page asks with HTTP 402 and a PAYMENT-REQUIRED header; retry with PAYMENT-SIGNATURE (x402 v2).
Bulk copies
Bulk copies are available to AI training crawlers for a fee via x402. Each download is a separate payment for the current version of the file. The terms are in /x402/catalog.json. The lead-magnet sheet (/five-jobs/the-five-jobs-sheet.pdf) is not for sale. Write to AGMM.ltd@gmail.com for anything else.
Unknown bots
Free and watched. Nothing is charged on a guess. Measured only: nobody is charged. We count how many such requests there are. A payment is never taken for a page that does not exist.
Licence
What a payment buys, and what it does not, is on the licence page.
Machine-readable
/crawlers.json has the same policy as JSON. Paid delivery count is public and verifiable on-chain.