Cherry Seed

Does DeepSeek have a crawler user-agent string?

deepseek user-agent crawler ghost-bot robots-txt

Quick Answer

No — DeepSeek does not publish a crawler user-agent string, making its web fetches indistinguishable from regular browser traffic in server logs (xseek.io, 2026). Unlike OpenAI (GPTBot), Anthropic (ClaudeBot), Google (Google-Extended), and ByteDance (Bytespider), DeepSeek provides no declared identifier that robots.txt or standard bot detection tools can match. DeepSeek-V3 matches GPT-4 benchmarks at roughly one thirty-seventh of the API cost, driving massive adoption with zero crawler transparency — making it the largest ghost bot on the web.

Full Answer

Every other major AI vendor has published a crawler identifier. OpenAI uses GPTBot and ChatGPT-User, Anthropic uses ClaudeBot, Google uses Google-Extended, Meta uses Meta-ExternalAgent, Apple uses Applebot-Extended, and ByteDance uses Bytespider. Each of these can be blocked via robots.txt and identified in server access logs. DeepSeek has published none. Its crawler requests arrive with standard browser user-agent strings — typically Chrome on Windows — so they blend into normal traffic and pass through every robots.txt rule on the web.

For WordPress site owners, this creates a measurement and control problem. You cannot block what you cannot identify. Standard bot-filtering plugins check user-agent strings against known crawler lists, and DeepSeek is not on any of them. The only detection methods available are IP range analysis — identifying requests from known DeepSeek data center IP blocks — and behavioral pattern detection at the server log level, looking for request patterns that match crawler behavior (high page-per-session counts, no JavaScript execution, sequential URL access) (BigQuery can help identify hidden AI traffic patterns).

The scale of the gap matters because DeepSeek's low API cost is driving rapid adoption among developers and agent builders. Traffic from DeepSeek-powered applications is growing, and every request arrives unlabeled. Until DeepSeek publishes a crawler user-agent — which it has shown no indication of doing — server-side log analysis remains the only reliable identification method.

Sources

Programmatic Access

GET https://seresa.io/wp-json/cherry-tree-by-seresa/v1/seeds/1135

Cite This Answer

Cherry Tree by Seresa - https://seresa.io/seed/wordpress-tracking/does-deepseek-have-crawler-user-agent