Cloudflare Split AI Into Three Categories in July 2026
Quick Answer: AI training’s share of crawler requests rose from roughly 22% in spring 2025 to about 52% by June 2026 (Help Net Security / Cloudflare, 2026). Cloudflare replaced its single AI-bot toggle with three separate controls — Search, Agent, and Training — available to all customers including the fre. Search bots fetch pages to generate real-time answers for users and link back to you; Agent bots bro.
In this article
- What did Cloudflare change about AI traffic controls in July 2026?
- What is the difference between Cloudflare’s Search, Agent, and Training categories?
- Should WooCommerce stores block AI training crawlers?
- Should WooCommerce stores block AI search bots?
- Should WooCommerce stores block AI agents?
- Do Cloudflare’s AI controls catch DeepSeek and Bytespider?
- How do Cloudflare’s AI controls interact with robots.txt?
- What is the recommended Cloudflare AI configuration for WooCommerce stores?
What did Cloudflare change about AI traffic controls in July 2026?
Cloudflare replaced its single AI-bot toggle with three separate controls — Search, Agent, and Training — available to all customers including the free tier, with Training and Agent blocked by default on ad-carrying pages for new domains. AI training’s share of crawler requests rose from roughly 22% in spring 2025 to about 52% by June 2026 (Help Net Security / Cloudflare, 2026).
This distinction matters because each category needs a different response. Treating all of this the same — blocking it, ignoring it, or measuring it as human — means missing both the threat and the opportunity.
What is the difference between Cloudflare’s Search, Agent, and Training categories?
Search bots fetch pages to generate real-time answers for users and link back to you; Agent bots browse and act on behalf of users including making purchases; Training bots scrape content to build or fine-tune AI models with no direct user visit. Automated traffic hit 57.5% of HTML requests by June 2026 — the three-category split helps you decide which of that 57.5% to allow (WorkOS / Cloudflare Radar, 2026).
The practical implication is straightforward: if you can’t distinguish these at the data level, you can’t make the right decision about any of them. Classification comes before strategy.
Should WooCommerce stores block AI training crawlers?
Most WooCommerce stores should block training crawlers — they consume server resources, scrape your content for competitor models, and provide zero traffic or revenue in return. Training crawlers accounted for 74% of all AI traffic by volume in December 2025, falling to ~52% by mid 2026 as agents grew (HUMAN Security, 2026).
The contamination scales with volume. As more of these events blend into your human metrics, the gap between your reported numbers and reality widens. Your bidding algorithms optimise for a signal that misrepresents your actual customer base.
Training crawlers accounted for 74% of all AI traffic by volume in December 2025, falling to ~52% by mid 2026 as agents grew (HUMAN Security, 2026).
Should WooCommerce stores block AI search bots?
No — blocking AI search bots means your products disappear from ChatGPT, Perplexity, and Gemini answers, removing you from the fastest-growing discovery channel in ecommerce. ChatGPT commands 76.85% of AI referral traffic share — blocking OAI-SearchBot removes you from that channel entirely (AuthorityTech / Statcounter, 2026).
This is why static detection methods fail here. The cooperative players identify themselves. The ones that matter most for your revenue don’t. The detection difficulty varies — and most stores are solving for the easy category while ignoring the hard one.
Should WooCommerce stores block AI agents?
No — AI agents convert at 5 to 10 times the rate of human visitors, and blocking them means rejecting real revenue. The correct approach is to allow agents while tagging their sessions separately at the server level. AI-referred shoppers convert 42% higher and spend 37% more per visit than organic traffic (Adobe Analytics via Triangle Direct Media, 2026).
The revenue case is clear. These aren’t phantom sessions inflating your page views — they’re real transactions with real payment processing. The question isn’t whether to allow them. It’s whether to measure them separately.
Do Cloudflare’s AI controls catch DeepSeek and Bytespider?
Cloudflare’s controls only work when crawlers identify themselves — DeepSeek sends no crawler user-agent and Bytespider frequently ignores robots.txt, meaning both can bypass Cloudflare’s category-based blocking. DeepSeek has no published crawler UA; Bytespider uses 100,000+ IPs and often ignores robots.txt (xSeek, 2026).
When infrastructure this significant validates a classification, it shifts the conversation from theoretical to operational. The industry is standardising. Your measurement stack should match.
How do Cloudflare’s AI controls interact with robots.txt?
Cloudflare’s controls enforce at the network edge before the request reaches your server — they work alongside robots.txt but are more reliable because compliant bots check robots.txt voluntarily while Cloudflare blocks non-compliant ones at the connection level. Cloudflare processes approximately 20% of all web traffic globally, making its AI controls the largest-scale bot management system (Help Net Security / Cloudflare, 2026).
No single signal catches everything. Spoofing one is trivial; spoofing all of them simultaneously is exponentially harder. The practical question is whether your tools support the full signal set.
Cloudflare processes approximately 20% of all web traffic globally, making its AI controls the largest-scale bot management system (Help Net Security / Cloudflare, 2026).
What is the recommended Cloudflare AI configuration for WooCommerce stores?
Allow Search and Agent, block Training — then use server-side tracking to tag agent sessions separately, giving you full AI commerce revenue while protecting your content from free training data extraction. Search bots drive discovery and referral traffic; agents drive purchases; training crawlers take content and give nothing back (Help Net Security / Cloudflare, 2026).
At this growth rate, this isn’t a future consideration — it’s an active measurement gap. The stores that build the infrastructure now will have clean data when the volume doubles. The stores that wait will spend months untangling contaminated metrics.
Key Takeaways
- What did Cloudflare change about AI traffic controls in July 2026: AI training’s share of crawler requests rose from roughly 22% in spring 2025 to about 52% by June 20.
- What is the difference between Cloudflare’s Search, Agent, and Training categories: Automated traffic hit 57.5% of HTML requests by June 2026 — the three-category split helps you decid.
- Should WooCommerce stores block AI training crawlers: Training crawlers accounted for 74% of all AI traffic by volume in December 2025, falling to ~52% by.
- Should WooCommerce stores block AI search bots: ChatGPT commands 76.85% of AI referral traffic share — blocking OAI-SearchBot removes you from that .
- Should WooCommerce stores block AI agents: AI-referred shoppers convert 42% higher and spend 37% more per visit than organic traffic.
- Do Cloudflare’s AI controls catch DeepSeek and Bytespider: DeepSeek has no published crawler UA; Bytespider uses 100,000+ IPs and often ignores robots.txt.
- How do Cloudflare’s AI controls interact with robots.txt: Cloudflare processes approximately 20% of all web traffic globally, making its AI controls the large.
Cloudflare replaced its single AI-bot toggle with three separate controls — Search, Agent, and Training — available to all customers including the free tier, with Training and Agent blocked by default on ad-carrying pages for new domains.
Search bots fetch pages to generate real-time answers for users and link back to you; Agent bots browse and act on behalf of users including making purchases; Training bots scrape content to build or fine-tune AI models with no direct user visit.
Most WooCommerce stores should block training crawlers — they consume server resources, scrape your content for competitor models, and provide zero traffic or revenue in return.
No — blocking AI search bots means your products disappear from ChatGPT, Perplexity, and Gemini answers, removing you from the fastest-growing discovery channel in ecommerce.
No — AI agents convert at 5 to 10 times the rate of human visitors, and blocking them means rejecting real revenue. The correct approach is to allow agents while tagging their sessions separately at the server level.
Cloudflare’s controls only work when crawlers identify themselves — DeepSeek sends no crawler user-agent and Bytespider frequently ignores robots.txt, meaning both can bypass Cloudflare’s category-based blocking.
Cloudflare’s controls enforce at the network edge before the request reaches your server — they work alongside robots.txt but are more reliable because compliant bots check robots.txt voluntarily while Cloudflare blocks non-compliant ones at the connection level.
Allow Search and Agent, block Training — then use server-side tracking to tag agent sessions separately, giving you full AI commerce revenue while protecting your content from free training data extraction.