Full Answer

Bytespider is ByteDance's primary web crawler, built to feed training data into its AI models including Doubao and its family of large language models. Unlike Googlebot, which sends organic traffic back to the sites it indexes, Bytespider extracts content with no reciprocal value — your server does the work, and you get nothing measurable in return.

The scale is what makes it dangerous for WordPress sites specifically. With over 100,000 documented IP addresses and a pattern of ignoring robots.txt rules (Chinese AI Blocking Guide 2026), Bytespider can generate thousands of requests per day to a single site. WordPress processes every one through its full stack: PHP initialisation, database lookups, theme template rendering, and every active plugin hook. That's real CPU and memory per request, and at volume, it compounds into measurable performance degradation for your actual customers.

The most reliable way to block it is at the server level. A user-agent rule in your Nginx or Apache configuration stops the request before WordPress ever loads. If you're behind Cloudflare, their AI Crawlers category control blocks Bytespider (and other AI scrapers) with a single toggle. For a deeper look at which AI crawlers are hitting WordPress sites and whether they send any traffic back, see ChatGPT crawled your WordPress site — did it actually send you a customer?.