Yahoo Slurp
| Vendor | Yahoo (Verizon Media) |
| Type | Search crawler (legacy; Yahoo Search results increasingly come from Bing) |
| robots.txt token | Slurp |
| JavaScript rendering | Partial |
| Honors robots.txt | Yes |
User-Agent string
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)Purpose
The classic crawler for Yahoo! Search. In practice Yahoo's search results have been mostly Bing-derived since 2009 (US) and 2018 (Japan). The Slurp crawler still runs but its weight in actual rankings is small outside of Yahoo's regional properties.
The Japanese property yahoo.co.jp operates its own search and crawler infrastructure separately.
In our audit
In v1 we observed yahoo.com blocks GPTBot, ChatGPT-User, ClaudeBot, PerplexityBot. yahoo.co.jp goes further: it IP-walls real Chrome from our datacenter while allowing Googlebot, Bingbot, and Applebot UAs from the same IP — a classic UA-only allowlist pattern.
How to allow / block
User-agent: Slurp
Disallow: /