Cherry Seed

What is the difference between an AI crawler and a shopping agent?

ai-crawler shopping-agent search-bot gptbot training-crawler agentic-traffic

Quick Answer

AI crawlers scrape website content to train language models and identify themselves with known user-agent strings like GPTBot or ClaudeBot. Shopping agents browse and purchase products on behalf of human users, running real Chromium browsers with standard Chrome signatures indistinguishable from human visitors. HUMAN Security's 2026 report found training crawlers accounted for 74% of AI traffic by volume but generated zero revenue, while shopping agents represented just 1.7% of volume yet grew 7,851% year-over-year. Blocking crawlers saves server resources — blocking agents costs you sales.

Full Answer

The confusion starts because most analytics tools label everything non-human as "bot traffic" and filter it out. That's a problem when some of that traffic is placing orders. Training crawlers — GPTBot, ClaudeBot, Bytespider — send identifiable user-agent headers, respect robots.txt directives, and never interact with your cart or checkout. They're indexing your content for AI model training, not shopping. You can block them with a single robots.txt rule and lose nothing but bandwidth.

Shopping agents are fundamentally different. They run full Chromium browser instances, carry standard Chrome user-agent strings, and execute JavaScript — including your tracking scripts, if those scripts fire client-side. They add products to carts, fill shipping forms, and complete payment. Because their browser fingerprint matches a real visitor, your analytics has no reliable way to separate them from humans without server-side event inspection.

Seresa's breakdown of the six types of AI visitors your analytics mixes together (https://seresa.io/blog/ai-data-readiness/stop-calling-it-ai-traffic-the-six-types-of-ai-visitors-your-analytics-is-mixing-together) explains how these categories collapse in GA4 and why the distinction between a crawler and an agent is the difference between a cost centre and a revenue channel.

Sources

Programmatic Access

GET https://seresa.io/wp-json/cherry-tree-by-seresa/v1/seeds/1130

Cite This Answer

Cherry Tree by Seresa - https://seresa.io/seed/ai-data-readiness/difference-ai-crawler-search-bot-shopping-agent