Full Answer

Before July 2026, Cloudflare offered a single toggle: block all AI bots or allow all of them. That binary forced a bad trade-off — blocking everything meant losing AI search referral traffic, while allowing everything invited unchecked model training on your content.

The new three-category system resolves that. Search bots — from platforms like Google AI Overviews, ChatGPT search, and Perplexity — fetch your pages to answer user queries in real time and typically link back to the source. Agent bots go further: they browse, compare products, and complete purchases on a user's behalf. Both categories drive measurable traffic and revenue. Training bots, by contrast, scrape content in bulk to build or fine-tune language models, offering no immediate traffic or attribution in return.

Cloudflare's data shows training crawlers now dominate the mix, growing from roughly a fifth to over half of all AI crawler requests in just over a year. For WooCommerce stores, the recommended configuration is straightforward: allow Search and Agent, block Training. That preserves your visibility in AI-driven search results and keeps agent commerce channels open, without giving away content for free. The distinction also matters for analytics — understanding which category a bot falls into determines whether you should count, measure, or simply block it. For a deeper breakdown of how these categories differ from traditional crawlers, see What is the difference between an AI crawler and a shopping agent?.