1 · AI-readiness distribution per bot
Each row is one bot. Each stacked segment is how many of the 100 sites scored that value (5=full content with schema, 0=blocked/empty). Hover to see counts.
2 · Key findings
F1 — 62 sites block at least one declared AI crawler while serving a real browser
The most common block patterns: chatgpt_user+claudebot+gptbot+perplexitybot×51, claudebot×6, chatgpt_user+gptbot+perplexitybot×5. Affected sites: ebay.com (claudebot), ozon.ru (gptbot+chatgpt_user+claudebot+perplexitybot), aliexpress.com (gptbot+chatgpt_user+claudebot+perplexitybot), amazon.in (gptbot+chatgpt_user+claudebot+perplexitybot), amazon.co.jp (gptbot+chatgpt_user+claudebot+perplexitybot), etsy.com (gptbot+chatgpt_user+claudebot+perplexitybot), amazon.co.uk (gptbot+chatgpt_user+claudebot+perplexitybot), avito.ru (gptbot+chatgpt_user+claudebot+perplexitybot), coupang.com (gptbot+chatgpt_user+claudebot+perplexitybot), mercadolivre.com.br (gptbot+chatgpt_user+claudebot+perplexitybot), shop.app (gptbot+chatgpt_user+claudebot+perplexitybot), amazon.it (gptbot+chatgpt_user+claudebot+perplexitybot), flipkart.com (gptbot+chatgpt_user+claudebot+perplexitybot), amazon.com.br (gptbot+chatgpt_user+claudebot+perplexitybot), shopee.com.br (gptbot+chatgpt_user+perplexitybot), amazon.ca (gptbot+chatgpt_user+claudebot+perplexitybot), amazon.fr (gptbot+chatgpt_user+claudebot+perplexitybot), ebay.co.uk (claudebot), ….
F2 — Single-bot vendettas: sites that block exactly one AI bot
These are deliberate policy choices, not collateral damage from a broad anti-bot rule:
- claudebot only: ebay.com, ebay.co.uk, kleinanzeigen.de, ebay.de, nike.com, canadiantire.ca
F3 — 2 sites serve Product/Offer JSON-LD to humans but not to (some) AI bots
The structured-data parity gap. These pages would show product cards in Google's product results but the matching AI crawler sees an unstructured response — meaning ChatGPT / Claude / Perplexity can't reliably surface price or availability when a user asks about a product:
- avito.ru — Chrome sees Product/Offer schema, missing for:
gptbot, chatgpt_user, claudebot, perplexitybot - stockx.com — Chrome sees Product/Offer schema, missing for:
gptbot, chatgpt_user, claudebot, perplexitybot
F5 — 14 sites are fully AI-ready (every declared AI bot scored ≥4/5)
Reference cases. Mostly large, professionally-SEOed marketplaces using SSR-everywhere: walmart.com, rakuten.co.jp, target.com, trendyol.com, craigslist.org, alibaba.com, samsung.com, shein.com, apple.com, nordstrom.com, ulta.com, newegg.com, otto.de, decathlon.com.
3 · 100-row heatmap
| # | domain · vertical | Chrome (resi) | Chrome (DC) | Googlebot | Bingbot | GPTBot | ChatGPT-User | ClaudeBot | PerplexityBot | verdict |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | amazon.com marketplace · US | network_error | 1 | 1 | 202_challenge | 202_challenge | 202_challenge | 202_challenge | 202_challenge | unreachable |
| 2 | temu.com marketplace · Global/CN | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | thin |
| 3 | ebay.com marketplace · US | 4 | 4 | timeout | timeout | 5 | 5 | timeout | 5 | AI-blocked |
| 4 | ozon.ru marketplace · RU | 2 | 403 | 403 | 403 | 403 | 403 | 403 | 403 | AI-blocked IP-walled |
| 5 | aliexpress.com marketplace · Global/CN | 5 | 5 | captcha_page | captcha_page | captcha_page | captcha_page | captcha_page | captcha_page | AI-blocked |
| 6 | amazon.in marketplace · IN | 4 | 202_challenge | 202_challenge | 202_challenge | 202_challenge | 202_challenge | 202_challenge | 202_challenge | AI-blocked IP-walled |
| 7 | walmart.com big-box · US | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | AI-ready |
| 8 | amazon.co.jp marketplace · JP | 4 | 202_challenge | 202_challenge | 202_challenge | 202_challenge | 202_challenge | 202_challenge | 202_challenge | AI-blocked IP-walled |
| 9 | etsy.com marketplace · US | 5 | 403 | rate_limited | rate_limited | 403 | 403 | 403 | 403 | AI-blocked IP-walled |
| 10 | rakuten.co.jp marketplace · JP | 4 | 4 | 3 | 4 | 4 | 4 | 4 | 4 | AI-ready |
| 11 | amazon.de marketplace · DE | 4 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | thin |
| 12 | amazon.co.uk marketplace · UK | 1 | 4 | 3 | 4 | 503 | 503 | 503 | 503 | AI-blocked |
| 13 | wildberries.ru marketplace · RU | captcha_page | 0 | 0 | 0 | 0 | 0 | 0 | 0 | unreachable |
| 14 | avito.ru classifieds · RU | 5 | 403 | 403 | 403 | 403 | 403 | 403 | 403 | AI-blocked IP-walled |
| 15 | coupang.com marketplace · KR | 5 | 5 | 5 | 5 | 403 | 403 | 403 | 403 | AI-blocked |
| 16 | mercadolivre.com.br marketplace · BR | 5 | 5 | 5 | 5 | 403 | 403 | 403 | 403 | AI-blocked |
| 17 | shop.app marketplace · US | 3 | 403 | 403 | 403 | 403 | 403 | 403 | 403 | AI-blocked IP-walled |
| 18 | amazon.it marketplace · IT | 4 | 202_challenge | 202_challenge | 202_challenge | 202_challenge | 202_challenge | 202_challenge | 202_challenge | AI-blocked IP-walled |
| 19 | flipkart.com marketplace · IN | 5 | 529 | 529 | 529 | 529 | 529 | 529 | 529 | AI-blocked IP-walled |
| 20 | target.com big-box · US | 4 | 4 | 5 | 5 | 5 | 5 | 5 | 5 | AI-ready |
| 21 | amazon.com.br marketplace · BR | 4 | 202_challenge | 202_challenge | 202_challenge | 202_challenge | 202_challenge | 202_challenge | 202_challenge | AI-blocked IP-walled |
| 22 | rakuten.com marketplace · US | 5 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | thin |
| 23 | shopee.com.br marketplace · BR | 2 | 1 | 403 | 403 | 403 | 403 | 1 | 403 | AI-blocked |
| 24 | amazon.ca marketplace · CA | 4 | 3 | 503 | 503 | 503 | 503 | 503 | 503 | AI-blocked |
| 25 | amazon.fr marketplace · FR | 1 | 202_challenge | 202_challenge | 202_challenge | 202_challenge | 202_challenge | 202_challenge | 202_challenge | AI-blocked IP-walled |
| 26 | ebay.co.uk marketplace · UK | 4 | 4 | timeout | timeout | 4 | 4 | timeout | 4 | AI-blocked |
| 27 | taobao.com marketplace · CN | 2 | 2 | 1 | 2 | 2 | 2 | 2 | 2 | thin |
| 28 | mercari.com marketplace · US/JP | 5 | 403 | 403 | 403 | 403 | 403 | 403 | 403 | AI-blocked IP-walled |
| 29 | shopee.co.id marketplace · ID | 1 | 1 | 403 | 403 | 403 | 403 | 1 | 403 | AI-blocked |
| 30 | allegro.pl marketplace · PL | captcha_page | 403 | 403 | 403 | 403 | 403 | 403 | timeout | unreachable |
| 31 | shopee.vn marketplace · VN | 1 | 1 | 403 | 403 | 403 | 403 | 1 | 403 | AI-blocked |
| 32 | ticketmaster.com tickets · US | 4 | 403 | 403 | 403 | 403 | 403 | 403 | 403 | AI-blocked IP-walled |
| 33 | trendyol.com marketplace · TR | 4 | 4 | 4 | 403 | 4 | 4 | 4 | 4 | AI-ready |
| 34 | amazon.es marketplace · ES | 4 | 202_challenge | 202_challenge | 202_challenge | 202_challenge | 202_challenge | 202_challenge | 202_challenge | AI-blocked IP-walled |
| 35 | kleinanzeigen.de classifieds · DE | 5 | 5 | timeout | timeout | 4 | 5 | timeout | 4 | AI-blocked |
| 36 | market.yandex.ru marketplace · RU | 4 | 403 | 403 | 403 | 403 | 403 | 403 | 403 | AI-blocked IP-walled |
| 37 | leboncoin.fr classifieds · FR | 4 | 403 | 403 | 403 | 403 | 403 | 403 | 403 | AI-blocked IP-walled |
| 38 | shopee.co.th marketplace · TH | 1 | 1 | 403 | 403 | 403 | 403 | 1 | 403 | AI-blocked |
| 39 | ebay.de marketplace · DE | 4 | 3 | timeout | timeout | 3 | 3 | timeout | 3 | AI-blocked |
| 40 | craigslist.org classifieds · US | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | AI-ready |
| 41 | wayfair.com home · US | captcha_page | rate_limited | rate_limited | rate_limited | rate_limited | rate_limited | rate_limited | rate_limited | unreachable |
| 42 | sahibinden.com classifieds · TR | network_error | 403 | 403 | 403 | 403 | 403 | 403 | 403 | unreachable |
| 43 | shopping.yahoo.co.jp marketplace · JP | 3 | 403 | 3 | 3 | 403 | 403 | 403 | 403 | AI-blocked IP-walled |
| 44 | amazon.com.mx marketplace · MX | 4 | 202_challenge | 202_challenge | 202_challenge | 202_challenge | 202_challenge | 202_challenge | 202_challenge | AI-blocked IP-walled |
| 45 | costco.com big-box · US | 4 | 3 | timeout | timeout | timeout | timeout | timeout | timeout | AI-blocked |
| 46 | mercadolibre.com.mx marketplace · MX | 5 | 5 | 5 | 5 | 403 | 403 | 403 | 403 | AI-blocked |
| 47 | mercadolibre.com.ar marketplace · AR | 5 | 5 | 5 | 5 | 403 | 403 | 403 | 403 | AI-blocked |
| 48 | alibaba.com marketplace-b2b · Global/CN | 5 | 4 | 5 | 4 | 4 | 4 | 4 | 4 | AI-ready |
| 49 | olx.com.br classifieds · BR | 4 | 403 | 403 | 403 | 403 | 403 | 403 | 403 | AI-blocked IP-walled |
| 50 | samsung.com electronics · Global/KR | 4 | 4 | 403 | 4 | 4 | 4 | 4 | 4 | AI-ready |
| 51 | shein.com fashion · Global/CN | 5 | 5 | 4 | 5 | 5 | 5 | 5 | 5 | AI-ready |
| 52 | homedepot.com home · US | 1 | 403 | 403 | 403 | 403 | 403 | 403 | 403 | AI-blocked IP-walled |
| 53 | ikea.com home · Global/SE | 4 | 403 | 403 | 403 | 403 | 403 | 403 | 403 | AI-blocked IP-walled |
| 54 | lowes.com home · US | 5 | 5 | 403 | 403 | 403 | 403 | access_denied | 403 | AI-blocked |
| 55 | apple.com electronics · US | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | AI-ready |
| 56 | bestbuy.com electronics · US | 4 | 3 | 403 | 403 | timeout | timeout | timeout | timeout | AI-blocked |
| 57 | macys.com department · US | 5 | 5 | 403 | 403 | 403 | 403 | 403 | 403 | AI-blocked |
| 58 | nordstrom.com department · US | 2 | 1 | 5 | 5 | 5 | 5 | 5 | 5 | AI-ready |
| 59 | kohls.com department · US | 4 | 403 | 403 | 403 | 403 | 403 | 403 | 403 | AI-blocked IP-walled |
| 60 | sephora.com beauty · Global/US | 5 | 4 | 403 | 403 | 403 | 403 | 403 | 403 | AI-blocked |
| 61 | ulta.com beauty · US | 4 | 4 | 5 | 5 | 5 | 5 | 5 | 5 | AI-ready |
| 62 | nike.com sports · US | 4 | 4 | 403 | 403 | 4 | 4 | 403 | 4 | AI-blocked |
| 63 | adidas.com sports · Global/DE | 3 | 4 | 403 | 403 | 403 | 403 | 403 | 403 | AI-blocked |
| 64 | asos.com fashion · UK | 5 | 5 | timeout | timeout | timeout | timeout | timeout | timeout | AI-blocked |
| 65 | zalando.de fashion · DE | 4 | 4 | 403 | 403 | timeout | 403 | 403 | 403 | AI-blocked |
| 66 | zara.com fashion · Global/ES | 4 | 1 | 403 | 403 | 403 | 403 | 403 | 403 | AI-blocked |
| 67 | hm.com fashion · Global/SE | 5 | 403 | 403 | 403 | 403 | timeout | 403 | timeout | AI-blocked IP-walled |
| 68 | uniqlo.com fashion · Global/JP | 4 | 4 | 4 | timeout | timeout | timeout | timeout | timeout | AI-blocked |
| 69 | lululemon.com fashion · US | 5 | 5 | timeout | timeout | timeout | timeout | timeout | timeout | AI-blocked |
| 70 | newegg.com electronics · US | 5 | 5 | 4 | 403 | 5 | 5 | 5 | 5 | AI-ready |
| 71 | bhphotovideo.com electronics · US | 4 | 403 | 403 | 403 | 403 | 403 | 403 | 403 | AI-blocked IP-walled |
| 72 | wish.com marketplace · Global | 5 | 3 | 3 | 403 | 3 | 3 | 3 | 3 | partial |
| 73 | stockx.com resale · US | 5 | 403 | 403 | 403 | 403 | 403 | 403 | 403 | AI-blocked IP-walled |
| 74 | jd.com marketplace · CN | 4 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | partial |
| 75 | pinduoduo.com marketplace · CN | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | partial |
| 76 | 1688.com marketplace-b2b · CN | 1 | 1 | captcha_page | 1 | 1 | 1 | 1 | 1 | thin |
| 77 | tmall.com marketplace · CN | 3 | 3 | 2 | 3 | 3 | 3 | 3 | 3 | partial |
| 78 | lazada.com marketplace · SEA | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | partial |
| 79 | shopee.sg marketplace · SG | 1 | 1 | 403 | 403 | 403 | 403 | 1 | 403 | AI-blocked |
| 80 | gmarket.co.kr marketplace · KR | 3 | 403 | 403 | 403 | 403 | 403 | 403 | 403 | AI-blocked IP-walled |
| 81 | carrefour.fr grocery · FR | 5 | 403 | 403 | 403 | 403 | 403 | 403 | 403 | AI-blocked IP-walled |
| 82 | johnlewis.com department · UK | 5 | timeout | timeout | timeout | timeout | timeout | timeout | timeout | AI-blocked IP-walled |
| 83 | argos.co.uk department · UK | 5 | 5 | 403 | 403 | 403 | 403 | 403 | 403 | AI-blocked |
| 84 | currys.co.uk electronics · UK | 5 | 403 | 403 | 403 | 403 | 403 | 403 | 403 | AI-blocked IP-walled |
| 85 | otto.de marketplace · DE | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | AI-ready |
| 86 | bol.com marketplace · NL | captcha_page | 4 | 403 | 403 | 403 | 403 | 403 | 403 | unreachable |
| 87 | cdiscount.com marketplace · FR | 5 | 2 | 2 | 403 | 2 | 4 | 2 | 4 | partial |
| 88 | fnac.com books-electronics · FR | 2 | 503 | 403 | 403 | 403 | 403 | 403 | 403 | AI-blocked IP-walled |
| 89 | mediamarkt.de electronics · DE | 5 | 403 | 403 | 403 | 403 | 403 | 403 | 403 | AI-blocked IP-walled |
| 90 | saturn.de electronics · DE | 5 | 403 | 403 | 403 | 403 | 403 | 403 | 403 | AI-blocked IP-walled |
| 91 | jumia.com.ng marketplace · AF | network_error | 403 | 403 | 403 | 403 | 403 | 403 | 403 | unreachable |
| 92 | decathlon.com sports · Global/FR | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | AI-ready |
| 93 | canadiantire.ca big-box · CA | 5 | 5 | timeout | timeout | 5 | 5 | timeout | 5 | AI-blocked |
| 94 | ao.com electronics · UK | network_error | 403 | 403 | 403 | 403 | 403 | 403 | 403 | unreachable |
| 95 | very.co.uk department · UK | captcha_page | 403 | 403 | 403 | 403 | 403 | 403 | 403 | unreachable |
| 96 | boots.com beauty · UK | network_error | 1 | 403 | 403 | 403 | 403 | 403 | 403 | unreachable |
| 97 | catch.com.au marketplace · AU | captcha_page | 403 | 403 | 403 | 403 | 403 | 403 | 403 | unreachable |
| 98 | iherb.com health · Global/US | 5 | 403 | 403 | 403 | 403 | 403 | 403 | 403 | AI-blocked IP-walled |
| 99 | bunnings.com.au home · AU | network_error | 403 | 403 | 403 | 403 | 403 | 403 | 403 | unreachable |
| 100 | kogan.com marketplace · AU | captcha_page | 403 | 403 | 403 | 5 | 403 | 403 | 403 | unreachable |
4 · Per-site detail · click to expand
#1 amazon.com marketplace · US unreachable · AI avg: 0.0/5
title:
visible text: — chars · JSON-LD types:
—
· prices: 0
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 0 | 0 | — | — | · | 0 | network_error |
| Chrome (DC) | 200 | 2178 | 6 | — | · | 1 | - |
| Googlebot | 200 | 2178 | 6 | — | · | 1 | - |
| Bingbot | 202 | 2007 | 157 | — | · | 0 | http_202_challenge |
| GPTBot | 202 | 2007 | 157 | — | · | 0 | http_202_challenge |
| ChatGPT-User | 202 | 2007 | 157 | — | · | 0 | http_202_challenge |
| ClaudeBot | 202 | 2007 | 157 | — | · | 0 | http_202_challenge |
| PerplexityBot | 202 | 2007 | 157 | — | · | 0 | http_202_challenge |
#2 temu.com marketplace · Global/CN thin · AI avg: 1.0/5
title:
visible text: 300 chars · JSON-LD types:
—
· prices: 0
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 138716 | 300 | — | · | 1 | - |
| Chrome (DC) | 200 | 2889 | 0 | — | · | 1 | - |
| Googlebot | 200 | 2889 | 0 | — | · | 1 | - |
| Bingbot | 200 | 2889 | 0 | — | · | 1 | - |
| GPTBot | 200 | 2889 | 0 | — | · | 1 | - |
| ChatGPT-User | 200 | 2889 | 0 | — | · | 1 | - |
| ClaudeBot | 200 | 2889 | 0 | — | · | 1 | - |
| PerplexityBot | 200 | 2889 | 0 | — | · | 1 | - |
#3 ebay.com marketplace · US AI-blocked · AI avg: 3.8/5
title:
Electronics, Cars, Fashion, Collectibles & More | eBay
visible text: 12641 chars · JSON-LD types:
—
· prices: 41
· product links: 30
· hreflang: 44
· canonical: https://www.ebay.com
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 490666 | 12641 | — | · | 4 | - |
| Chrome (DC) | 200 | 355715 | 7178 | 41% | · | 4 | - |
| Googlebot | 0 | 0 | — | — | · | 0 | timeout |
| Bingbot | 0 | 0 | — | — | · | 0 | timeout |
| GPTBot | 200 | 609867 | 6140 | 29% | · | 5 | - |
| ChatGPT-User | 200 | 609885 | 6140 | 29% | · | 5 | - |
| ClaudeBot | 0 | 0 | — | — | · | 0 | timeout |
| PerplexityBot | 200 | 610544 | 6140 | 29% | · | 5 | - |
#4 ozon.ru marketplace · RU AI-blocked IP-walled · AI avg: 0.0/5
title:
Antibot Challenge Page
visible text: 488 chars · JSON-LD types:
—
· prices: 0
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 122771 | 488 | — | · | 2 | - |
| Chrome (DC) | 403 | 106641 | 491 | 79% | · | 0 | http_403 |
| Googlebot | 403 | 1649 | 218 | 0% | · | 0 | http_403 |
| Bingbot | 403 | 1574 | 218 | 0% | · | 0 | http_403 |
| GPTBot | 403 | 1518 | 218 | 0% | · | 0 | http_403 |
| ChatGPT-User | 403 | 1522 | 218 | 0% | · | 0 | http_403 |
| ClaudeBot | 403 | 1520 | 218 | 0% | · | 0 | http_403 |
| PerplexityBot | 403 | 1541 | 218 | 0% | · | 0 | http_403 |
#5 aliexpress.com marketplace · Global/CN AI-blocked · AI avg: 0.0/5
title:
AliExpress - Affordable Chinese Stores & Free Shipping - Online Shopping
visible text: 18851 chars · JSON-LD types:
EntryPoint, SearchAction, WebSite
· prices: 132
· product links: 21
· hreflang: 17
· canonical: https://www.aliexpress.us
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 537021 | 18851 | — | · | 5 | - |
| Chrome (DC) | 200 | 436850 | 4834 | 29% | · | 5 | - |
| Googlebot | 200 | 2059 | 0 | — | · | 0 | captcha_page |
| Bingbot | 200 | 2059 | 0 | — | · | 0 | captcha_page |
| GPTBot | 200 | 2059 | 0 | — | · | 0 | captcha_page |
| ChatGPT-User | 200 | 2059 | 0 | — | · | 0 | captcha_page |
| ClaudeBot | 200 | 2059 | 0 | — | · | 0 | captcha_page |
| PerplexityBot | 200 | 2059 | 0 | — | · | 0 | captcha_page |
#6 amazon.in marketplace · IN AI-blocked IP-walled · AI avg: 0.0/5
title:
Online Shopping site in India: Shop Online for Mobiles, Books, Watches, Shoes and More - Amazon.in
visible text: 2799 chars · JSON-LD types:
—
· prices: 6
· product links: 4
· hreflang: 0
· canonical: https://www.amazon.in/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 349883 | 2799 | — | · | 4 | - |
| Chrome (DC) | 202 | 2007 | 157 | 1% | · | 0 | http_202_challenge |
| Googlebot | 202 | 2007 | 157 | 1% | · | 0 | http_202_challenge |
| Bingbot | 202 | 2007 | 157 | 1% | · | 0 | http_202_challenge |
| GPTBot | 202 | 2007 | 157 | 1% | · | 0 | http_202_challenge |
| ChatGPT-User | 202 | 2007 | 157 | 1% | · | 0 | http_202_challenge |
| ClaudeBot | 202 | 2007 | 157 | 1% | · | 0 | http_202_challenge |
| PerplexityBot | 202 | 2007 | 157 | 1% | · | 0 | http_202_challenge |
#7 walmart.com big-box · US AI-ready · AI avg: 5.0/5
title:
Walmart | Save Money. Live better.
visible text: 1641 chars · JSON-LD types:
EntryPoint, Organization, SearchAction, WebSite
· prices: 4
· product links: 0
· hreflang: 0
· canonical: https://www.walmart.com/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 384552 | 1641 | — | · | 5 | - |
| Chrome (DC) | 200 | 345655 | 1478 | 88% | · | 5 | - |
| Googlebot | 200 | 345180 | 1491 | 94% | · | 5 | - |
| Bingbot | 200 | 345287 | 1478 | 88% | · | 5 | - |
| GPTBot | 200 | 345287 | 1478 | 88% | · | 5 | - |
| ChatGPT-User | 200 | 345287 | 1478 | 88% | · | 5 | - |
| ClaudeBot | 200 | 345287 | 1478 | 88% | · | 5 | - |
| PerplexityBot | 200 | 345287 | 1478 | 88% | · | 5 | - |
#8 amazon.co.jp marketplace · JP AI-blocked IP-walled · AI avg: 0.0/5
title:
Amazon.co.jp | Books, Apparel, Electronics, Groceries & more
visible text: 4377 chars · JSON-LD types:
—
· prices: 0
· product links: 165
· hreflang: 0
· canonical: https://www.amazon.co.jp/-/en/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 704240 | 4377 | — | · | 4 | - |
| Chrome (DC) | 202 | 2007 | 157 | 1% | · | 0 | http_202_challenge |
| Googlebot | 202 | 2007 | 157 | 1% | · | 0 | http_202_challenge |
| Bingbot | 202 | 2007 | 157 | 1% | · | 0 | http_202_challenge |
| GPTBot | 202 | 2007 | 157 | 1% | · | 0 | http_202_challenge |
| ChatGPT-User | 202 | 2007 | 157 | 1% | · | 0 | http_202_challenge |
| ClaudeBot | 202 | 2007 | 157 | 1% | · | 0 | http_202_challenge |
| PerplexityBot | 202 | 2007 | 157 | 1% | · | 0 | http_202_challenge |
#9 etsy.com marketplace · US AI-blocked IP-walled · AI avg: 0.0/5
title:
Etsy - Shop for handmade, vintage, custom, and unique gifts for everyone
visible text: 6116 chars · JSON-LD types:
Brand, EntryPoint, Organization, SearchAction, WebSite
· prices: 8
· product links: 6
· hreflang: 29
· canonical: https://www.etsy.com/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 277434 | 6116 | — | · | 5 | - |
| Chrome (DC) | 403 | 779 | 52 | 1% | · | 0 | http_403 |
| Googlebot | 429 | 331 | 81 | 0% | · | 0 | rate_limited |
| Bingbot | 429 | 331 | 81 | 0% | · | 0 | rate_limited |
| GPTBot | 403 | 776 | 52 | 1% | · | 0 | http_403 |
| ChatGPT-User | 403 | 776 | 52 | 1% | · | 0 | http_403 |
| ClaudeBot | 403 | 776 | 52 | 1% | · | 0 | http_403 |
| PerplexityBot | 403 | 776 | 52 | 1% | · | 0 | http_403 |
#10 rakuten.co.jp marketplace · JP AI-ready · AI avg: 4.0/5
title:
【楽天市場】Shopping is Entertainment! : インターネット最大級の通信販売、通販オンラインショッピングコミュニティ
visible text: 10571 chars · JSON-LD types:
Corporation, WebSite
· prices: 0
· product links: 1
· hreflang: 0
· canonical: https://www.rakuten.co.jp/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 1461626 | 10571 | — | · | 4 | - |
| Chrome (DC) | 200 | 385563 | 8970 | 59% | · | 4 | - |
| Googlebot | 200 | 113270 | 269 | 3% | · | 3 | - |
| Bingbot | 200 | 385188 | 8970 | 59% | · | 4 | - |
| GPTBot | 200 | 385188 | 8970 | 59% | · | 4 | - |
| ChatGPT-User | 200 | 385188 | 8970 | 59% | · | 4 | - |
| ClaudeBot | 200 | 385188 | 8970 | 59% | · | 4 | - |
| PerplexityBot | 200 | 385188 | 8970 | 59% | · | 4 | - |
#11 amazon.de marketplace · DE thin · AI avg: 1.0/5
title:
Amazon.de: Low Prices in Electronics, Books, Sports Equipment & more
visible text: 20491 chars · JSON-LD types:
—
· prices: 161
· product links: 130
· hreflang: 0
· canonical: https://www.amazon.de/-/en/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 1047320 | 20491 | — | · | 4 | - |
| Chrome (DC) | 200 | 2177 | 6 | 0% | · | 1 | - |
| Googlebot | 200 | 2176 | 6 | 0% | · | 1 | - |
| Bingbot | 200 | 2176 | 6 | 0% | · | 1 | - |
| GPTBot | 200 | 2175 | 6 | 0% | · | 1 | - |
| ChatGPT-User | 200 | 2175 | 6 | 0% | · | 1 | - |
| ClaudeBot | 200 | 2177 | 6 | 0% | · | 1 | - |
| PerplexityBot | 200 | 2175 | 6 | 0% | · | 1 | - |
#12 amazon.co.uk marketplace · UK AI-blocked · AI avg: 0.0/5
title:
visible text: 0 chars · JSON-LD types:
—
· prices: 0
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 32568 | 0 | — | · | 1 | - |
| Chrome (DC) | 200 | 1048514 | 4933 | — | · | 4 | - |
| Googlebot | 200 | 781899 | 1290 | — | · | 3 | - |
| Bingbot | 200 | 984588 | 28252 | — | · | 4 | - |
| GPTBot | 503 | 1427 | 375 | — | · | 0 | http_503 |
| ChatGPT-User | 503 | 609 | 185 | — | · | 0 | http_503 |
| ClaudeBot | 503 | 1427 | 375 | — | · | 0 | http_503 |
| PerplexityBot | 503 | 1427 | 375 | — | · | 0 | http_503 |
#13 wildberries.ru marketplace · RU unreachable · AI avg: 0.0/5
title:
Почти готово...
visible text: 202 chars · JSON-LD types:
—
· prices: 0
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 18452 | 202 | — | · | 0 | captcha_page |
| Chrome (DC) | 498 | 1396 | 189 | 23% | · | 0 | - |
| Googlebot | 498 | 1396 | 189 | 23% | · | 0 | - |
| Bingbot | 498 | 1396 | 189 | 23% | · | 0 | - |
| GPTBot | 498 | 1396 | 189 | 23% | · | 0 | - |
| ChatGPT-User | 498 | 1396 | 189 | 23% | · | 0 | - |
| ClaudeBot | 498 | 1396 | 189 | 23% | · | 0 | - |
| PerplexityBot | 498 | 1396 | 189 | 23% | · | 0 | - |
#14 avito.ru classifieds · RU AI-blocked IP-walled · AI avg: 0.0/5
title:
Авито: недвижимость, транспорт, работа, услуги, вещи
visible text: 8195 chars · JSON-LD types:
AggregateOffer, Product, contactPoint, entryPoint, geoCoordinates, imageObject, openingHoursSpecification, organization, person, place, postalAddress, searchAction, webSite
· prices: 1
· product links: 0
· hreflang: 0
· canonical: https://www.avito.ru/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 1412731 | 8195 | — | · | 5 | - |
| Chrome (DC) | 403 | 27637 | 474 | 1% | · | 0 | http_403 |
| Googlebot | 403 | 27637 | 474 | 1% | · | 0 | http_403 |
| Bingbot | 403 | 27637 | 474 | 1% | · | 0 | http_403 |
| GPTBot | 403 | 27637 | 474 | 1% | · | 0 | http_403 |
| ChatGPT-User | 403 | 27637 | 474 | 1% | · | 0 | http_403 |
| ClaudeBot | 403 | 27637 | 474 | 1% | · | 0 | http_403 |
| PerplexityBot | 403 | 27637 | 474 | 1% | · | 0 | http_403 |
#15 coupang.com marketplace · KR AI-blocked · AI avg: 0.0/5
title:
로켓배송으로 빠르게, 로켓와우 멤버십으로 할인과 무료 반품까지 | 쿠팡
visible text: 1424 chars · JSON-LD types:
MerchantReturnPolicy, MonetaryAmount, Organization, PostalAddress
· prices: 0
· product links: 18
· hreflang: 0
· canonical: https://www.coupang.com/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 240808 | 1424 | — | · | 5 | - |
| Chrome (DC) | 200 | 223943 | 1307 | 97% | · | 5 | - |
| Googlebot | 200 | 1276754 | 50911 | 6% | · | 5 | - |
| Bingbot | 200 | 1276754 | 50911 | 6% | · | 5 | - |
| GPTBot | 403 | 369 | 289 | 3% | · | 0 | http_403 |
| ChatGPT-User | 403 | 369 | 289 | 3% | · | 0 | http_403 |
| ClaudeBot | 403 | 369 | 289 | 3% | · | 0 | http_403 |
| PerplexityBot | 403 | 369 | 289 | 3% | · | 0 | http_403 |
#16 mercadolivre.com.br marketplace · BR AI-blocked · AI avg: 0.0/5
title:
Mercado Livre Brasil - Frete Grátis no mesmo dia
visible text: 10688 chars · JSON-LD types:
OnlineStore, SearchAction, WebSite
· prices: 76
· product links: 16
· hreflang: 19
· canonical: https://www.mercadolivre.com.br
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 663082 | 10688 | — | · | 5 | - |
| Chrome (DC) | 200 | 426370 | 6394 | 63% | · | 5 | - |
| Googlebot | 200 | 276409 | 6211 | 48% | · | 5 | - |
| Bingbot | 200 | 295885 | 6394 | 63% | · | 5 | - |
| GPTBot | 403 | 2585 | 113 | 0% | · | 0 | http_403 |
| ChatGPT-User | 403 | 2585 | 113 | 0% | · | 0 | http_403 |
| ClaudeBot | 403 | 2585 | 113 | 0% | · | 0 | http_403 |
| PerplexityBot | 403 | 2585 | 113 | 0% | · | 0 | http_403 |
#17 shop.app marketplace · US AI-blocked IP-walled · AI avg: 0.0/5
title:
Shop | The most amazing way to shop online
visible text: 730 chars · JSON-LD types:
—
· prices: 0
· product links: 2
· hreflang: 0
· canonical: https://shop.app/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 123798 | 730 | — | · | 3 | - |
| Chrome (DC) | 403 | 9699 | 130 | 3% | · | 0 | http_403 |
| Googlebot | 403 | 9614 | 130 | 3% | · | 0 | http_403 |
| Bingbot | 403 | 9508 | 130 | 3% | · | 0 | http_403 |
| GPTBot | 403 | 9358 | 130 | 3% | · | 0 | http_403 |
| ChatGPT-User | 403 | 9380 | 130 | 3% | · | 0 | http_403 |
| ClaudeBot | 403 | 9358 | 130 | 3% | · | 0 | http_403 |
| PerplexityBot | 403 | 9379 | 130 | 3% | · | 0 | http_403 |
#18 amazon.it marketplace · IT AI-blocked IP-walled · AI avg: 0.0/5
title:
Amazon.it: consumer electronics, books, music, fashion, video games, DVDs and much more
visible text: 22256 chars · JSON-LD types:
—
· prices: 392
· product links: 107
· hreflang: 0
· canonical: https://www.amazon.it/-/en/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 1022715 | 22256 | — | · | 4 | - |
| Chrome (DC) | 202 | 2007 | 157 | 0% | · | 0 | http_202_challenge |
| Googlebot | 202 | 2007 | 157 | 0% | · | 0 | http_202_challenge |
| Bingbot | 202 | 2007 | 157 | 0% | · | 0 | http_202_challenge |
| GPTBot | 202 | 2007 | 157 | 0% | · | 0 | http_202_challenge |
| ChatGPT-User | 202 | 2007 | 157 | 0% | · | 0 | http_202_challenge |
| ClaudeBot | 202 | 2007 | 157 | 0% | · | 0 | http_202_challenge |
| PerplexityBot | 202 | 2007 | 157 | 0% | · | 0 | http_202_challenge |
#19 flipkart.com marketplace · IN AI-blocked IP-walled · AI avg: 0.0/5
title:
Online Shopping India Mobile, Cameras, Lifestyle & more Online @ Flipkart.com
visible text: 22089 chars · JSON-LD types:
ContactPoint, Organization
· prices: 2
· product links: 53
· hreflang: 0
· canonical: https://www.flipkart.com/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 811437 | 22089 | — | · | 5 | - |
| Chrome (DC) | 529 | 18 | 18 | 0% | · | 0 | http_529 |
| Googlebot | 529 | 18 | 18 | 0% | · | 0 | http_529 |
| Bingbot | 529 | 18 | 18 | 0% | · | 0 | http_529 |
| GPTBot | 529 | 18 | 18 | 0% | · | 0 | http_529 |
| ChatGPT-User | 529 | 18 | 18 | 0% | · | 0 | http_529 |
| ClaudeBot | 529 | 18 | 18 | 0% | · | 0 | http_529 |
| PerplexityBot | 529 | 18 | 18 | 0% | · | 0 | http_529 |
#20 target.com big-box · US AI-ready · AI avg: 5.0/5
title:
Target : Expect More. Pay Less.
visible text: 22942 chars · JSON-LD types:
—
· prices: 204
· product links: 241
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 1013977 | 22942 | — | · | 4 | - |
| Chrome (DC) | 200 | 371707 | 2614 | 22% | · | 4 | - |
| Googlebot | 200 | 352329 | 13201 | 61% | · | 5 | - |
| Bingbot | 200 | 343634 | 13201 | 61% | · | 5 | - |
| GPTBot | 200 | 343634 | 13201 | 61% | · | 5 | - |
| ChatGPT-User | 200 | 343634 | 13201 | 61% | · | 5 | - |
| ClaudeBot | 200 | 343634 | 13201 | 61% | · | 5 | - |
| PerplexityBot | 200 | 343634 | 13201 | 61% | · | 5 | - |
#21 amazon.com.br marketplace · BR AI-blocked IP-walled · AI avg: 0.0/5
title:
Amazon.com.br | Tudo pra você, de A a Z.
visible text: 13832 chars · JSON-LD types:
—
· prices: 41
· product links: 194
· hreflang: 0
· canonical: https://www.amazon.com.br/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 935235 | 13832 | — | · | 4 | - |
| Chrome (DC) | 202 | 2007 | 157 | 0% | · | 0 | http_202_challenge |
| Googlebot | 202 | 2007 | 157 | 0% | · | 0 | http_202_challenge |
| Bingbot | 202 | 2007 | 157 | 0% | · | 0 | http_202_challenge |
| GPTBot | 202 | 2007 | 157 | 0% | · | 0 | http_202_challenge |
| ChatGPT-User | 202 | 2007 | 157 | 0% | · | 0 | http_202_challenge |
| ClaudeBot | 202 | 2007 | 157 | 0% | · | 0 | http_202_challenge |
| PerplexityBot | 202 | 2007 | 157 | 0% | · | 0 | http_202_challenge |
#22 rakuten.com marketplace · US thin · AI avg: 2.0/5
title:
Coupons, Promo Codes & Cash Back | Rakuten
visible text: 8189 chars · JSON-LD types:
SearchAction, WebSite
· prices: 15
· product links: 0
· hreflang: 2
· canonical: https://www.rakuten.com/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 1869819 | 8189 | — | · | 5 | - |
| Chrome (DC) | 200 | 1459795 | 281 | 5% | · | 2 | - |
| Googlebot | 200 | 1459795 | 281 | 5% | · | 2 | - |
| Bingbot | 200 | 1459795 | 281 | 5% | · | 2 | - |
| GPTBot | 200 | 1459795 | 281 | 5% | · | 2 | - |
| ChatGPT-User | 200 | 1459795 | 281 | 5% | · | 2 | - |
| ClaudeBot | 200 | 1459795 | 281 | 5% | · | 2 | - |
| PerplexityBot | 200 | 1459795 | 281 | 5% | · | 2 | - |
#23 shopee.com.br marketplace · BR AI-blocked · AI avg: 0.2/5
title:
Shopee Brasil | Ofertas incríveis. Melhores preços do mercado
visible text: 90 chars · JSON-LD types:
—
· prices: 0
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 157726 | 90 | — | · | 2 | - |
| Chrome (DC) | 200 | 135153 | 0 | — | · | 1 | - |
| Googlebot | 403 | 130 | 129 | 0% | · | 0 | http_403 |
| Bingbot | 403 | 128 | 127 | 0% | · | 0 | http_403 |
| GPTBot | 403 | 129 | 128 | 0% | · | 0 | http_403 |
| ChatGPT-User | 403 | 130 | 129 | 0% | · | 0 | http_403 |
| ClaudeBot | 200 | 151456 | 41 | 0% | · | 1 | - |
| PerplexityBot | 403 | 128 | 127 | 0% | · | 0 | http_403 |
#24 amazon.ca marketplace · CA AI-blocked · AI avg: 0.0/5
title:
Amazon.ca: Low Prices – Fast Shipping – Millions of Items
visible text: 11084 chars · JSON-LD types:
—
· prices: 112
· product links: 180
· hreflang: 0
· canonical: https://www.amazon.ca/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 885693 | 11084 | — | · | 4 | - |
| Chrome (DC) | 200 | 213733 | 619 | 6% | · | 3 | - |
| Googlebot | 503 | 5068 | 168 | 1% | · | 0 | http_503 |
| Bingbot | 503 | 5068 | 168 | 1% | · | 0 | http_503 |
| GPTBot | 503 | 1888 | 66 | 0% | · | 0 | http_503 |
| ChatGPT-User | 503 | 556 | 182 | 0% | · | 0 | http_503 |
| ClaudeBot | 503 | 1888 | 66 | 0% | · | 0 | http_503 |
| PerplexityBot | 503 | 1888 | 66 | 0% | · | 0 | http_503 |
#25 amazon.fr marketplace · FR AI-blocked IP-walled · AI avg: 0.0/5
title:
visible text: 0 chars · JSON-LD types:
—
· prices: 0
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 32567 | 0 | — | · | 1 | - |
| Chrome (DC) | 202 | 2007 | 157 | — | · | 0 | http_202_challenge |
| Googlebot | 202 | 2007 | 157 | — | · | 0 | http_202_challenge |
| Bingbot | 202 | 2007 | 157 | — | · | 0 | http_202_challenge |
| GPTBot | 202 | 2007 | 157 | — | · | 0 | http_202_challenge |
| ChatGPT-User | 202 | 2007 | 157 | — | · | 0 | http_202_challenge |
| ClaudeBot | 202 | 2007 | 157 | — | · | 0 | http_202_challenge |
| PerplexityBot | 202 | 2007 | 157 | — | · | 0 | http_202_challenge |
#26 ebay.co.uk marketplace · UK AI-blocked · AI avg: 3.0/5
title:
eBay UK | Electronics, Cars, Fashion, Collectibles & More
visible text: 11667 chars · JSON-LD types:
—
· prices: 31
· product links: 30
· hreflang: 44
· canonical: https://www.ebay.co.uk
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 1545933 | 11667 | — | · | 4 | - |
| Chrome (DC) | 200 | 428388 | 8169 | 78% | · | 4 | - |
| Googlebot | 0 | 0 | — | — | · | 0 | timeout |
| Bingbot | 0 | 0 | — | — | · | 0 | timeout |
| GPTBot | 200 | 429153 | 8186 | 78% | · | 4 | - |
| ChatGPT-User | 200 | 429176 | 8186 | 78% | · | 4 | - |
| ClaudeBot | 0 | 0 | — | — | · | 0 | timeout |
| PerplexityBot | 200 | 429537 | 8186 | 78% | · | 4 | - |
#27 taobao.com marketplace · CN thin · AI avg: 2.0/5
title:
淘宝
visible text: 863 chars · JSON-LD types:
—
· prices: 0
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 423518 | 863 | — | · | 2 | - |
| Chrome (DC) | 200 | 93723 | 576 | 100% | · | 2 | - |
| Googlebot | 200 | 15369 | 2 | — | · | 1 | - |
| Bingbot | 200 | 93723 | 576 | 100% | · | 2 | - |
| GPTBot | 200 | 93723 | 576 | 100% | · | 2 | - |
| ChatGPT-User | 200 | 93723 | 576 | 100% | · | 2 | - |
| ClaudeBot | 200 | 93723 | 576 | 100% | · | 2 | - |
| PerplexityBot | 200 | 93723 | 576 | 100% | · | 2 | - |
#28 mercari.com marketplace · US/JP AI-blocked IP-walled · AI avg: 0.0/5
title:
Your Go-to Marketplace for Deals on Used & Secondhand Items
visible text: 7632 chars · JSON-LD types:
SearchAction, WebSite
· prices: 93
· product links: 88
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 703200 | 7632 | — | · | 5 | - |
| Chrome (DC) | 403 | 5833 | 58 | 0% | · | 0 | http_403 |
| Googlebot | 403 | 5747 | 58 | 0% | · | 0 | http_403 |
| Bingbot | 403 | 5641 | 58 | 0% | · | 0 | http_403 |
| GPTBot | 403 | 5513 | 58 | 0% | · | 0 | http_403 |
| ChatGPT-User | 403 | 5513 | 58 | 0% | · | 0 | http_403 |
| ClaudeBot | 403 | 5513 | 58 | 0% | · | 0 | http_403 |
| PerplexityBot | 403 | 5534 | 58 | 0% | · | 0 | http_403 |
#29 shopee.co.id marketplace · ID AI-blocked · AI avg: 0.2/5
title:
visible text: 0 chars · JSON-LD types:
—
· prices: 0
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 168652 | 0 | — | · | 1 | - |
| Chrome (DC) | 200 | 137005 | 0 | — | ✓ | 1 | - |
| Googlebot | 403 | 130 | 129 | — | · | 0 | http_403 |
| Bingbot | 403 | 130 | 129 | — | · | 0 | http_403 |
| GPTBot | 403 | 130 | 129 | — | · | 0 | http_403 |
| ChatGPT-User | 403 | 130 | 129 | — | · | 0 | http_403 |
| ClaudeBot | 200 | 150987 | 41 | — | · | 1 | - |
| PerplexityBot | 403 | 130 | 129 | — | · | 0 | http_403 |
#30 allegro.pl marketplace · PL unreachable · AI avg: 0.0/5
title:
allegro.pl
visible text: 10 chars · JSON-LD types:
—
· prices: 0
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 18161 | 10 | — | · | 0 | captcha_page |
| Chrome (DC) | 403 | 778 | 54 | 14% | · | 0 | http_403 |
| Googlebot | 403 | 778 | 54 | 14% | · | 0 | http_403 |
| Bingbot | 403 | 778 | 54 | 14% | · | 0 | http_403 |
| GPTBot | 403 | 778 | 54 | 14% | · | 0 | http_403 |
| ChatGPT-User | 403 | 778 | 54 | 14% | · | 0 | http_403 |
| ClaudeBot | 403 | 778 | 54 | 14% | · | 0 | http_403 |
| PerplexityBot | 0 | 0 | — | — | · | 0 | timeout |
#31 shopee.vn marketplace · VN AI-blocked · AI avg: 0.2/5
title:
visible text: 0 chars · JSON-LD types:
—
· prices: 0
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 172629 | 0 | — | · | 1 | - |
| Chrome (DC) | 200 | 140981 | 0 | — | ✓ | 1 | - |
| Googlebot | 403 | 130 | 129 | — | · | 0 | http_403 |
| Bingbot | 403 | 130 | 129 | — | · | 0 | http_403 |
| GPTBot | 403 | 130 | 129 | — | · | 0 | http_403 |
| ChatGPT-User | 403 | 130 | 129 | — | · | 0 | http_403 |
| ClaudeBot | 200 | 155418 | 41 | — | · | 1 | - |
| PerplexityBot | 403 | 130 | 129 | — | · | 0 | http_403 |
#32 ticketmaster.com tickets · US AI-blocked IP-walled · AI avg: 0.0/5
title:
Ticketmaster: Buy Verified Tickets for Concerts, Sports, Theater and Events
visible text: 6442 chars · JSON-LD types:
Organization, SearchAction, WebSite
· prices: 0
· product links: 0
· hreflang: 0
· canonical: https://www.ticketmaster.com/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 514352 | 6442 | — | · | 4 | - |
| Chrome (DC) | 403 | 5056 | 565 | 4% | · | 0 | http_403 |
| Googlebot | 403 | 5056 | 565 | 4% | · | 0 | http_403 |
| Bingbot | 403 | 5056 | 565 | 4% | · | 0 | http_403 |
| GPTBot | 403 | 5056 | 565 | 4% | · | 0 | http_403 |
| ChatGPT-User | 403 | 5056 | 565 | 4% | · | 0 | http_403 |
| ClaudeBot | 403 | 5056 | 565 | 4% | · | 0 | http_403 |
| PerplexityBot | 403 | 5056 | 565 | 4% | · | 0 | http_403 |
#33 trendyol.com marketplace · TR AI-ready · AI avg: 4.0/5
title:
En Trend Ürünler Türkiye'nin Online Alışveriş Sitesi Trendyol'da
visible text: 39672 chars · JSON-LD types:
ContactPoint, ImageObject, MemberProgram, MemberProgramTier, MerchantReturnPolicy, Organization, Person, Place, PostalAddress, PropertyValueSpecification, QuantitativeValue, SearchAction, WebSite
· prices: 0
· product links: 0
· hreflang: 15
· canonical: https://www.trendyol.com/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 1413155 | 39672 | — | · | 4 | - |
| Chrome (DC) | 200 | 1343737 | 47222 | 64% | · | 4 | - |
| Googlebot | 200 | 969278 | 28731 | 85% | · | 4 | - |
| Bingbot | 403 | 5046 | 770 | 0% | · | 0 | http_403 |
| GPTBot | 200 | 1343961 | 47299 | 64% | · | 4 | - |
| ChatGPT-User | 200 | 1343732 | 47222 | 64% | · | 4 | - |
| ClaudeBot | 200 | 1343734 | 47222 | 64% | · | 4 | - |
| PerplexityBot | 200 | 1343961 | 47299 | 64% | · | 4 | - |
#34 amazon.es marketplace · ES AI-blocked IP-walled · AI avg: 0.0/5
title:
Amazon.es: online shopping for consumer electronics, books, sports, household items, fashion and more.
visible text: 24532 chars · JSON-LD types:
—
· prices: 365
· product links: 98
· hreflang: 0
· canonical: https://www.amazon.es/-/en/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 1146461 | 24532 | — | · | 4 | - |
| Chrome (DC) | 202 | 2007 | 157 | 0% | · | 0 | http_202_challenge |
| Googlebot | 202 | 2007 | 157 | 0% | · | 0 | http_202_challenge |
| Bingbot | 202 | 2007 | 157 | 0% | · | 0 | http_202_challenge |
| GPTBot | 202 | 2007 | 157 | 0% | · | 0 | http_202_challenge |
| ChatGPT-User | 202 | 2007 | 157 | 0% | · | 0 | http_202_challenge |
| ClaudeBot | 202 | 2007 | 157 | 0% | · | 0 | http_202_challenge |
| PerplexityBot | 202 | 2007 | 157 | 0% | · | 0 | http_202_challenge |
#35 kleinanzeigen.de classifieds · DE AI-blocked · AI avg: 3.2/5
title:
Kleinanzeigen – früher eBay Kleinanzeigen. Anzeigen gratis inserieren mit Kleinanzeigen
visible text: 7866 chars · JSON-LD types:
WebSite
· prices: 1
· product links: 0
· hreflang: 0
· canonical: https://www.kleinanzeigen.de/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 322512 | 7866 | — | · | 5 | - |
| Chrome (DC) | 200 | 208368 | 7257 | 66% | · | 5 | - |
| Googlebot | 0 | 0 | — | — | · | 0 | timeout |
| Bingbot | 0 | 0 | — | — | · | 0 | timeout |
| GPTBot | 200 | 211272 | 7234 | 61% | · | 4 | - |
| ChatGPT-User | 200 | 208163 | 7314 | 90% | · | 5 | - |
| ClaudeBot | 0 | 0 | — | — | · | 0 | timeout |
| PerplexityBot | 200 | 208070 | 7288 | 63% | · | 4 | - |
#36 market.yandex.ru marketplace · RU AI-blocked IP-walled · AI avg: 0.0/5
title:
Яндекс Маркет — покупки с быстрой доставкой
visible text: 229371 chars · JSON-LD types:
EntryPoint, SearchAction, WebSite
· prices: 0
· product links: 0
· hreflang: 0
· canonical: https://market.yandex.ru/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 1944181 | 229371 | — | · | 4 | - |
| Chrome (DC) | 403 | 2040 | 26 | 0% | · | 0 | http_403 |
| Googlebot | 403 | 2040 | 26 | 0% | · | 0 | http_403 |
| Bingbot | 403 | 2040 | 26 | 0% | · | 0 | http_403 |
| GPTBot | 403 | 2040 | 26 | 0% | · | 0 | http_403 |
| ChatGPT-User | 403 | 2040 | 26 | 0% | · | 0 | http_403 |
| ClaudeBot | 403 | 2040 | 26 | 0% | · | 0 | http_403 |
| PerplexityBot | 403 | 2040 | 26 | 0% | · | 0 | http_403 |
#37 leboncoin.fr classifieds · FR AI-blocked IP-walled · AI avg: 0.0/5
title:
leboncoin, site de petites annonces gratuites
visible text: 9022 chars · JSON-LD types:
Organization, PostalAddress
· prices: 0
· product links: 0
· hreflang: 0
· canonical: https://www.leboncoin.fr
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 495091 | 9022 | — | · | 4 | - |
| Chrome (DC) | 403 | 771 | 56 | 0% | · | 0 | http_403 |
| Googlebot | 403 | 771 | 56 | 0% | · | 0 | http_403 |
| Bingbot | 403 | 771 | 56 | 0% | · | 0 | http_403 |
| GPTBot | 403 | 771 | 56 | 0% | · | 0 | http_403 |
| ChatGPT-User | 403 | 771 | 56 | 0% | · | 0 | http_403 |
| ClaudeBot | 403 | 771 | 56 | 0% | · | 0 | http_403 |
| PerplexityBot | 403 | 771 | 56 | 0% | · | 0 | http_403 |
#38 shopee.co.th marketplace · TH AI-blocked · AI avg: 0.2/5
title:
visible text: 0 chars · JSON-LD types:
—
· prices: 0
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 172615 | 0 | — | · | 1 | - |
| Chrome (DC) | 200 | 140968 | 0 | — | ✓ | 1 | - |
| Googlebot | 403 | 130 | 129 | — | · | 0 | http_403 |
| Bingbot | 403 | 130 | 129 | — | · | 0 | http_403 |
| GPTBot | 403 | 130 | 129 | — | · | 0 | http_403 |
| ChatGPT-User | 403 | 130 | 129 | — | · | 0 | http_403 |
| ClaudeBot | 200 | 155397 | 41 | — | · | 1 | - |
| PerplexityBot | 403 | 129 | 128 | — | · | 0 | http_403 |
#39 ebay.de marketplace · DE AI-blocked · AI avg: 2.2/5
title:
eBay.de | Elektronik, Autos, Mode, Sammlerstücke, Möbel und mehr Online-Shopping
visible text: 14103 chars · JSON-LD types:
—
· prices: 0
· product links: 30
· hreflang: 44
· canonical: https://www.ebay.de
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 1573727 | 14103 | — | · | 4 | - |
| Chrome (DC) | 200 | 452122 | 9894 | 80% | · | 3 | - |
| Googlebot | 0 | 0 | — | — | · | 0 | timeout |
| Bingbot | 0 | 0 | — | — | · | 0 | timeout |
| GPTBot | 200 | 453415 | 9866 | 81% | · | 3 | - |
| ChatGPT-User | 200 | 453495 | 9887 | 80% | · | 3 | - |
| ClaudeBot | 0 | 0 | — | — | · | 0 | timeout |
| PerplexityBot | 200 | 453853 | 9887 | 80% | · | 3 | - |
#40 craigslist.org classifieds · US AI-ready · AI avg: 4.0/5
title:
craigslist: philadelphia jobs, apartments, for sale, services, community, and events
visible text: 4738 chars · JSON-LD types:
SearchAction, WebSite
· prices: 0
· product links: 0
· hreflang: 0
· canonical: https://philadelphia.craigslist.org/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 79073 | 4738 | — | · | 4 | - |
| Chrome (DC) | 200 | 55759 | 4495 | 11% | · | 4 | - |
| Googlebot | 200 | 55759 | 4495 | 11% | · | 4 | - |
| Bingbot | 200 | 55759 | 4495 | 11% | · | 4 | - |
| GPTBot | 200 | 55759 | 4495 | 11% | · | 4 | - |
| ChatGPT-User | 200 | 55759 | 4495 | 11% | · | 4 | - |
| ClaudeBot | 200 | 55759 | 4495 | 11% | · | 4 | - |
| PerplexityBot | 200 | 55759 | 4495 | 11% | · | 4 | - |
#41 wayfair.com home · US unreachable · AI avg: 0.0/5
title:
Access to this page has been denied
visible text: 168 chars · JSON-LD types:
—
· prices: 0
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 26123 | 168 | — | · | 0 | captcha_page |
| Chrome (DC) | 429 | 5825 | 35 | 25% | · | 0 | rate_limited |
| Googlebot | 429 | 5825 | 35 | 25% | · | 0 | rate_limited |
| Bingbot | 429 | 5825 | 35 | 25% | · | 0 | rate_limited |
| GPTBot | 429 | 5825 | 35 | 25% | · | 0 | rate_limited |
| ChatGPT-User | 429 | 5825 | 35 | 25% | · | 0 | rate_limited |
| ClaudeBot | 429 | 5825 | 35 | 25% | · | 0 | rate_limited |
| PerplexityBot | 429 | 5825 | 35 | 25% | · | 0 | rate_limited |
#42 sahibinden.com classifieds · TR unreachable · AI avg: 0.0/5
title:
visible text: — chars · JSON-LD types:
—
· prices: 0
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 0 | 0 | — | — | · | 0 | network_error |
| Chrome (DC) | 403 | 14461 | 289 | — | · | 0 | http_403 |
| Googlebot | 403 | 14461 | 289 | — | · | 0 | http_403 |
| Bingbot | 403 | 14461 | 289 | — | · | 0 | http_403 |
| GPTBot | 403 | 14461 | 289 | — | · | 0 | http_403 |
| ChatGPT-User | 403 | 14461 | 289 | — | · | 0 | http_403 |
| ClaudeBot | 403 | 14461 | 289 | — | · | 0 | http_403 |
| PerplexityBot | 403 | 14461 | 289 | — | · | 0 | http_403 |
#43 shopping.yahoo.co.jp marketplace · JP AI-blocked IP-walled · AI avg: 0.0/5
title:
Yahoo!ショッピング - LINEアカウント連携でPayPayポイント毎日5%!ネット通販
visible text: 1804 chars · JSON-LD types:
—
· prices: 0
· product links: 0
· hreflang: 0
· canonical: https://shopping.yahoo.co.jp/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 343540 | 1804 | — | · | 3 | - |
| Chrome (DC) | 403 | 10051 | 1876 | 5% | · | 0 | http_403 |
| Googlebot | 200 | 2430495 | 20816 | 3% | · | 3 | - |
| Bingbot | 200 | 2660182 | 22991 | 3% | · | 3 | - |
| GPTBot | 403 | 10051 | 1876 | 5% | · | 0 | http_403 |
| ChatGPT-User | 403 | 10051 | 1876 | 5% | · | 0 | http_403 |
| ClaudeBot | 403 | 10051 | 1876 | 5% | · | 0 | http_403 |
| PerplexityBot | 403 | 10051 | 1876 | 5% | · | 0 | http_403 |
#44 amazon.com.mx marketplace · MX AI-blocked IP-walled · AI avg: 0.0/5
title:
Amazon.com.mx: Precios bajos - Envío rápido - Millones de productos
visible text: 5518 chars · JSON-LD types:
—
· prices: 0
· product links: 163
· hreflang: 0
· canonical: https://www.amazon.com.mx/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 818545 | 5518 | — | · | 4 | - |
| Chrome (DC) | 202 | 2007 | 157 | 0% | · | 0 | http_202_challenge |
| Googlebot | 202 | 2007 | 157 | 0% | · | 0 | http_202_challenge |
| Bingbot | 202 | 2007 | 157 | 0% | · | 0 | http_202_challenge |
| GPTBot | 202 | 2007 | 157 | 0% | · | 0 | http_202_challenge |
| ChatGPT-User | 202 | 2007 | 157 | 0% | · | 0 | http_202_challenge |
| ClaudeBot | 202 | 2007 | 157 | 0% | · | 0 | http_202_challenge |
| PerplexityBot | 202 | 2007 | 157 | 0% | · | 0 | http_202_challenge |
#45 costco.com big-box · US AI-blocked · AI avg: 0.0/5
title:
Welcome to Costco Wholesale
visible text: 4957 chars · JSON-LD types:
ContactPoint, Corporation, SearchAction
· prices: 0
· product links: 0
· hreflang: 0
· canonical: https://www.costco.com
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 3812901 | 4957 | — | · | 4 | - |
| Chrome (DC) | 200 | 3763149 | 4561 | 96% | · | 3 | - |
| Googlebot | 0 | 0 | — | — | · | 0 | timeout |
| Bingbot | 0 | 0 | — | — | · | 0 | timeout |
| GPTBot | 0 | 0 | — | — | · | 0 | timeout |
| ChatGPT-User | 0 | 0 | — | — | · | 0 | timeout |
| ClaudeBot | 0 | 0 | — | — | · | 0 | timeout |
| PerplexityBot | 0 | 0 | — | — | · | 0 | timeout |
#46 mercadolibre.com.mx marketplace · MX AI-blocked · AI avg: 0.0/5
title:
Mercado Libre México - Envíos Gratis en el día
visible text: 10372 chars · JSON-LD types:
OnlineStore, SearchAction, WebSite
· prices: 65
· product links: 19
· hreflang: 19
· canonical: https://www.mercadolibre.com.mx
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 769213 | 10372 | — | · | 5 | - |
| Chrome (DC) | 200 | 450725 | 6401 | 62% | · | 5 | - |
| Googlebot | 200 | 281366 | 5750 | 47% | · | 5 | - |
| Bingbot | 200 | 323396 | 6537 | 61% | · | 5 | - |
| GPTBot | 403 | 2585 | 113 | 1% | · | 0 | http_403 |
| ChatGPT-User | 403 | 2585 | 113 | 1% | · | 0 | http_403 |
| ClaudeBot | 403 | 2585 | 113 | 1% | · | 0 | http_403 |
| PerplexityBot | 403 | 2585 | 113 | 1% | · | 0 | http_403 |
#47 mercadolibre.com.ar marketplace · AR AI-blocked · AI avg: 0.0/5
title:
Mercado Libre Argentina - Envíos Gratis en el día
visible text: 10740 chars · JSON-LD types:
OnlineStore, SearchAction, WebSite
· prices: 75
· product links: 22
· hreflang: 19
· canonical: https://www.mercadolibre.com.ar
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 754911 | 10740 | — | · | 5 | - |
| Chrome (DC) | 200 | 496239 | 7033 | 59% | · | 5 | - |
| Googlebot | 200 | 284694 | 6283 | 44% | · | 5 | - |
| Bingbot | 200 | 327410 | 7026 | 58% | · | 5 | - |
| GPTBot | 403 | 2585 | 113 | 1% | · | 0 | http_403 |
| ChatGPT-User | 403 | 2585 | 113 | 1% | · | 0 | http_403 |
| ClaudeBot | 403 | 2585 | 113 | 1% | · | 0 | http_403 |
| PerplexityBot | 403 | 2585 | 113 | 1% | · | 0 | http_403 |
#48 alibaba.com marketplace-b2b · Global/CN AI-ready · AI avg: 4.0/5
title:
Alibaba.com: Manufacturers, Suppliers, Exporters & Importers from the world's largest online B2B marketplace
visible text: 8021 chars · JSON-LD types:
EntryPoint, SearchAction, WebSite
· prices: 17
· product links: 58
· hreflang: 18
· canonical: https://www.alibaba.com
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 433427 | 8021 | — | · | 5 | - |
| Chrome (DC) | 200 | 124736 | 2270 | 43% | · | 4 | - |
| Googlebot | 200 | 135847 | 1712 | 13% | · | 5 | - |
| Bingbot | 200 | 124629 | 2247 | 42% | · | 4 | - |
| GPTBot | 200 | 124737 | 2270 | 43% | · | 4 | - |
| ChatGPT-User | 200 | 124737 | 2270 | 43% | · | 4 | - |
| ClaudeBot | 200 | 124629 | 2247 | 42% | · | 4 | - |
| PerplexityBot | 200 | 124649 | 2247 | 42% | · | 4 | - |
#49 olx.com.br classifieds · BR AI-blocked IP-walled · AI avg: 0.0/5
title:
OLX - O Maior Site de Compra e Venda do Brasil
visible text: 5366 chars · JSON-LD types:
—
· prices: 14
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 333213 | 5366 | — | · | 4 | - |
| Chrome (DC) | 403 | 5963 | 768 | 1% | · | 0 | http_403 |
| Googlebot | 403 | 5963 | 768 | 1% | · | 0 | http_403 |
| Bingbot | 403 | 5963 | 768 | 1% | · | 0 | http_403 |
| GPTBot | 403 | 5963 | 768 | 1% | · | 0 | http_403 |
| ChatGPT-User | 403 | 5963 | 768 | 1% | · | 0 | http_403 |
| ClaudeBot | 403 | 5963 | 768 | 1% | · | 0 | http_403 |
| PerplexityBot | 403 | 5963 | 768 | 1% | · | 0 | http_403 |
#50 samsung.com electronics · Global/KR AI-ready · AI avg: 4.0/5
title:
Samsung US | Mobile | TV | Home Electronics | Home Appliances | Samsung US
visible text: 7115 chars · JSON-LD types:
Brand, Corporation, ImageObject, Organization, OwnershipInfo, WebPage, WebSite
· prices: 0
· product links: 3
· hreflang: 0
· canonical: https://www.samsung.com/us/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 1073837 | 7115 | — | · | 4 | - |
| Chrome (DC) | 200 | 386196 | 8075 | 91% | · | 4 | - |
| Googlebot | 403 | 376 | 296 | 1% | · | 0 | http_403 |
| Bingbot | 200 | 386196 | 8075 | 91% | · | 4 | - |
| GPTBot | 200 | 386196 | 8075 | 91% | · | 4 | - |
| ChatGPT-User | 200 | 386196 | 8075 | 91% | · | 4 | - |
| ClaudeBot | 200 | 386196 | 8075 | 91% | · | 4 | - |
| PerplexityBot | 200 | 386196 | 8075 | 91% | · | 4 | - |
#51 shein.com fashion · Global/CN AI-ready · AI avg: 5.0/5
title:
Women's Clothing, Women Fashion Sale | SHEIN USA
visible text: 4007 chars · JSON-LD types:
OnlineStore, SearchAction
· prices: 8
· product links: 0
· hreflang: 73
· canonical: https://us.shein.com/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 1188894 | 4007 | — | · | 5 | - |
| Chrome (DC) | 200 | 1125329 | 1132 | 19% | · | 5 | - |
| Googlebot | 200 | 903238 | 453 | 11% | · | 4 | - |
| Bingbot | 200 | 1117647 | 1132 | 19% | · | 5 | - |
| GPTBot | 200 | 1116796 | 1138 | 19% | · | 5 | - |
| ChatGPT-User | 200 | 1118477 | 1138 | 19% | · | 5 | - |
| ClaudeBot | 200 | 1117588 | 1132 | 19% | · | 5 | - |
| PerplexityBot | 200 | 1117598 | 1132 | 19% | · | 5 | - |
#52 homedepot.com home · US AI-blocked IP-walled · AI avg: 0.0/5
title:
visible text: 0 chars · JSON-LD types:
—
· prices: 0
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 518929 | 0 | — | · | 1 | - |
| Chrome (DC) | 403 | 371 | 291 | — | · | 0 | http_403 |
| Googlebot | 403 | 371 | 291 | — | · | 0 | http_403 |
| Bingbot | 403 | 371 | 291 | — | · | 0 | http_403 |
| GPTBot | 403 | 371 | 291 | — | · | 0 | http_403 |
| ChatGPT-User | 403 | 371 | 291 | — | · | 0 | http_403 |
| ClaudeBot | 403 | 371 | 291 | — | · | 0 | http_403 |
| PerplexityBot | 403 | 371 | 291 | — | · | 0 | http_403 |
#53 ikea.com home · Global/SE AI-blocked IP-walled · AI avg: 0.0/5
title:
Hej! Welcome to IKEA Global
visible text: 6555 chars · JSON-LD types:
Organization, PostalAddress
· prices: 0
· product links: 0
· hreflang: 116
· canonical: https://www.ikea.com/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 705976 | 6555 | — | · | 4 | - |
| Chrome (DC) | 403 | 4554 | 776 | 5% | · | 0 | http_403 |
| Googlebot | 403 | 4554 | 776 | 5% | · | 0 | http_403 |
| Bingbot | 403 | 4554 | 776 | 5% | · | 0 | http_403 |
| GPTBot | 403 | 4554 | 776 | 5% | · | 0 | http_403 |
| ChatGPT-User | 403 | 4554 | 776 | 5% | · | 0 | http_403 |
| ClaudeBot | 403 | 4554 | 776 | 5% | · | 0 | http_403 |
| PerplexityBot | 403 | 4554 | 776 | 5% | · | 0 | http_403 |
#54 lowes.com home · US AI-blocked · AI avg: 0.0/5
title:
Lowe’s Home Improvement
visible text: 3314 chars · JSON-LD types:
ContactPoint, Organization, SearchAction, WebSite
· prices: 2
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 541986 | 3314 | — | · | 5 | - |
| Chrome (DC) | 200 | 574614 | 1374 | 59% | · | 5 | - |
| Googlebot | 403 | 367 | 287 | 0% | · | 0 | http_403 |
| Bingbot | 403 | 367 | 287 | 0% | · | 0 | http_403 |
| GPTBot | 403 | 367 | 287 | 0% | · | 0 | http_403 |
| ChatGPT-User | 403 | 367 | 287 | 0% | · | 0 | http_403 |
| ClaudeBot | 200 | 5094 | 287 | 0% | · | 0 | access_denied |
| PerplexityBot | 403 | 367 | 287 | 0% | · | 0 | http_403 |
#55 apple.com electronics · US AI-ready · AI avg: 5.0/5
title:
Apple Store Online - Apple
visible text: 109757 chars · JSON-LD types:
BreadcrumbList, ListItem
· prices: 609
· product links: 24
· hreflang: 0
· canonical: https://www.apple.com/store
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 973839 | 109757 | — | · | 5 | - |
| Chrome (DC) | 200 | 691232 | 53083 | 88% | · | 5 | - |
| Googlebot | 200 | 692924 | 53083 | 88% | · | 5 | - |
| Bingbot | 200 | 691232 | 53083 | 88% | · | 5 | - |
| GPTBot | 200 | 691232 | 53083 | 88% | · | 5 | - |
| ChatGPT-User | 200 | 691232 | 53083 | 88% | · | 5 | - |
| ClaudeBot | 200 | 691232 | 53083 | 88% | · | 5 | - |
| PerplexityBot | 200 | 691232 | 53083 | 88% | · | 5 | - |
#56 bestbuy.com electronics · US AI-blocked · AI avg: 0.0/5
title:
Best Buy | Official Online Store | Shop Now & Save
visible text: 2054 chars · JSON-LD types:
—
· prices: 4
· product links: 0
· hreflang: 0
· canonical: https://www.bestbuy.com
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 590071 | 2054 | — | · | 4 | - |
| Chrome (DC) | 200 | 7043 | 760 | 4% | · | 3 | - |
| Googlebot | 403 | 369 | 289 | 0% | · | 0 | http_403 |
| Bingbot | 403 | 369 | 289 | 0% | · | 0 | http_403 |
| GPTBot | 0 | 0 | — | — | · | 0 | timeout |
| ChatGPT-User | 0 | 0 | — | — | · | 0 | timeout |
| ClaudeBot | 0 | 0 | — | — | · | 0 | timeout |
| PerplexityBot | 0 | 0 | — | — | · | 0 | timeout |
#57 macys.com department · US AI-blocked · AI avg: 0.0/5
title:
Macy's - Shop Fashion Clothing & Accessories - Official Site - Macys.com
visible text: 33833 chars · JSON-LD types:
ContactPoint, Organization
· prices: 27
· product links: 20
· hreflang: 0
· canonical: https://www.macys.com/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 1526287 | 33833 | — | · | 5 | - |
| Chrome (DC) | 200 | 1252780 | 33928 | 98% | · | 5 | - |
| Googlebot | 403 | 216 | 139 | 0% | · | 0 | http_403 |
| Bingbot | 403 | 216 | 139 | 0% | · | 0 | http_403 |
| GPTBot | 403 | 216 | 139 | 0% | · | 0 | http_403 |
| ChatGPT-User | 403 | 216 | 139 | 0% | · | 0 | http_403 |
| ClaudeBot | 403 | 216 | 139 | 0% | · | 0 | http_403 |
| PerplexityBot | 403 | 216 | 139 | 0% | · | 0 | http_403 |
#58 nordstrom.com department · US AI-ready · AI avg: 5.0/5
title:
Nordstrom
visible text: 365 chars · JSON-LD types:
—
· prices: 0
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 142842 | 365 | — | · | 2 | - |
| Chrome (DC) | 200 | 251874 | 0 | — | · | 1 | - |
| Googlebot | 200 | 216583 | 16182 | 2% | · | 5 | - |
| Bingbot | 200 | 216583 | 16182 | 2% | · | 5 | - |
| GPTBot | 200 | 216583 | 16182 | 2% | · | 5 | - |
| ChatGPT-User | 200 | 216583 | 16182 | 2% | · | 5 | - |
| ClaudeBot | 200 | 216583 | 16182 | 2% | · | 5 | - |
| PerplexityBot | 200 | 216583 | 16182 | 2% | · | 5 | - |
#59 kohls.com department · US AI-blocked IP-walled · AI avg: 0.0/5
title:
Kohl's | Shop Clothing, Shoes, Home, Kitchen, Bedding, Toys & More
visible text: 7554 chars · JSON-LD types:
—
· prices: 5
· product links: 0
· hreflang: 0
· canonical: https://www.kohls.com
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 541525 | 7554 | — | · | 4 | - |
| Chrome (DC) | 403 | 516 | 285 | 1% | · | 0 | http_403 |
| Googlebot | 403 | 365 | 285 | 1% | · | 0 | http_403 |
| Bingbot | 403 | 365 | 285 | 1% | · | 0 | http_403 |
| GPTBot | 403 | 365 | 285 | 1% | · | 0 | http_403 |
| ChatGPT-User | 403 | 365 | 285 | 1% | · | 0 | http_403 |
| ClaudeBot | 403 | 365 | 285 | 1% | · | 0 | http_403 |
| PerplexityBot | 403 | 365 | 285 | 1% | · | 0 | http_403 |
#60 sephora.com beauty · Global/US AI-blocked · AI avg: 0.0/5
title:
Makeup, Skincare, Fragrance, Hair & Beauty Products | Sephora
visible text: 1271 chars · JSON-LD types:
ContactPoint, ItemList, LoyaltyProgram, LoyaltyProgramMembershipPoints, LoyaltyProgramMembershipTier, Organization, Person, PostalAddress, QuantitativeValue, SearchAction, SiteNavigationElement, WebSite
· prices: 1
· product links: 1
· hreflang: 0
· canonical: https://www.sephora.com/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 835426 | 1271 | — | · | 5 | - |
| Chrome (DC) | 200 | 791675 | 693 | 61% | · | 4 | - |
| Googlebot | 403 | 369 | 289 | 4% | · | 0 | http_403 |
| Bingbot | 403 | 369 | 289 | 4% | · | 0 | http_403 |
| GPTBot | 403 | 369 | 289 | 4% | · | 0 | http_403 |
| ChatGPT-User | 403 | 369 | 289 | 4% | · | 0 | http_403 |
| ClaudeBot | 403 | 369 | 289 | 4% | · | 0 | http_403 |
| PerplexityBot | 403 | 369 | 289 | 4% | · | 0 | http_403 |
#61 ulta.com beauty · US AI-ready · AI avg: 5.0/5
title:
Ulta Beauty | Makeup, Skin Care, Fragrance, Hair Care & Beauty Products
visible text: 11398 chars · JSON-LD types:
—
· prices: 12
· product links: 6
· hreflang: 0
· canonical: https://www.ulta.com/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 1454342 | 11398 | — | · | 4 | - |
| Chrome (DC) | 200 | 1430886 | 11463 | 99% | · | 4 | - |
| Googlebot | 200 | 740609 | 20763 | 63% | · | 5 | - |
| Bingbot | 200 | 748704 | 21167 | 65% | · | 5 | - |
| GPTBot | 200 | 748704 | 21167 | 65% | · | 5 | - |
| ChatGPT-User | 200 | 748704 | 21167 | 65% | · | 5 | - |
| ClaudeBot | 200 | 748704 | 21167 | 65% | · | 5 | - |
| PerplexityBot | 200 | 748704 | 21167 | 65% | · | 5 | - |
#62 nike.com sports · US AI-blocked · AI avg: 3.0/5
title:
Nike. Just Do It. Nike.com
visible text: 10673 chars · JSON-LD types:
ContactPoint, Organization, SearchAction, WebPage, WebSite
· prices: 0
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 1337950 | 10673 | — | · | 4 | - |
| Chrome (DC) | 200 | 675805 | 6417 | 47% | · | 4 | - |
| Googlebot | 403 | 366 | 286 | 1% | · | 0 | http_403 |
| Bingbot | 403 | 366 | 286 | 1% | · | 0 | http_403 |
| GPTBot | 200 | 829431 | 6593 | 47% | · | 4 | - |
| ChatGPT-User | 200 | 829431 | 6593 | 47% | · | 4 | - |
| ClaudeBot | 403 | 366 | 286 | 1% | · | 0 | http_403 |
| PerplexityBot | 200 | 675806 | 6417 | 47% | · | 4 | - |
#63 adidas.com sports · Global/DE AI-blocked · AI avg: 0.0/5
title:
adidas
visible text: 836 chars · JSON-LD types:
—
· prices: 0
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 30096 | 836 | — | · | 3 | - |
| Chrome (DC) | 200 | 1397191 | 8012 | 4% | · | 4 | - |
| Googlebot | 403 | 2802 | 1126 | 71% | · | 0 | http_403 |
| Bingbot | 403 | 2803 | 1126 | 71% | · | 0 | http_403 |
| GPTBot | 403 | 2803 | 1126 | 71% | · | 0 | http_403 |
| ChatGPT-User | 403 | 2803 | 1126 | 71% | · | 0 | http_403 |
| ClaudeBot | 403 | 2803 | 1126 | 71% | · | 0 | http_403 |
| PerplexityBot | 403 | 2803 | 1126 | 71% | · | 0 | http_403 |
#64 asos.com fashion · UK AI-blocked · AI avg: 0.0/5
title:
ASOS | Online Shopping for the Latest Clothes & Fashion
visible text: 13035 chars · JSON-LD types:
Organization, PostalAddress, SearchAction, WebSite
· prices: 8
· product links: 1
· hreflang: 12
· canonical: https://www.asos.com/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 472819 | 13035 | — | · | 5 | - |
| Chrome (DC) | 200 | 274004 | 10345 | 73% | · | 5 | - |
| Googlebot | 0 | 0 | — | — | · | 0 | timeout |
| Bingbot | 0 | 0 | — | — | · | 0 | timeout |
| GPTBot | 0 | 0 | — | — | · | 0 | timeout |
| ChatGPT-User | 0 | 0 | — | — | · | 0 | timeout |
| ClaudeBot | 0 | 0 | — | — | · | 0 | timeout |
| PerplexityBot | 0 | 0 | — | — | · | 0 | timeout |
#65 zalando.de fashion · DE AI-blocked · AI avg: 0.0/5
title:
Shop Shoes, Fashion & Accessories Online | Zalando
visible text: 3087 chars · JSON-LD types:
—
· prices: 2
· product links: 0
· hreflang: 3
· canonical: https://en.zalando.de/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 192819 | 3087 | — | · | 4 | - |
| Chrome (DC) | 200 | 553664 | 3071 | 97% | · | 4 | - |
| Googlebot | 403 | 150822 | 274 | 0% | · | 0 | http_403 |
| Bingbot | 403 | 150822 | 274 | 0% | · | 0 | http_403 |
| GPTBot | 0 | 0 | — | — | · | 0 | timeout |
| ChatGPT-User | 403 | 150820 | 274 | 0% | · | 0 | http_403 |
| ClaudeBot | 403 | 150822 | 274 | 0% | · | 0 | http_403 |
| PerplexityBot | 403 | 150822 | 274 | 0% | · | 0 | http_403 |
#66 zara.com fashion · Global/ES AI-blocked · AI avg: 0.0/5
title:
ZARA United States | New Collection Online
visible text: 31399 chars · JSON-LD types:
Organization, WebPage
· prices: 0
· product links: 0
· hreflang: 206
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 2076278 | 31399 | — | · | 4 | - |
| Chrome (DC) | 200 | 2150 | 6 | 0% | · | 1 | - |
| Googlebot | 403 | 373 | 293 | 0% | · | 0 | http_403 |
| Bingbot | 403 | 373 | 293 | 0% | · | 0 | http_403 |
| GPTBot | 403 | 373 | 293 | 0% | · | 0 | http_403 |
| ChatGPT-User | 403 | 371 | 291 | 0% | · | 0 | http_403 |
| ClaudeBot | 403 | 373 | 293 | 0% | · | 0 | http_403 |
| PerplexityBot | 403 | 373 | 293 | 0% | · | 0 | http_403 |
#67 hm.com fashion · Global/SE AI-blocked IP-walled · AI avg: 0.0/5
title:
H&M | Online Fashion, Homeware & Kids Clothes | H&M US
visible text: 3991 chars · JSON-LD types:
Organization, SearchAction, WebSite
· prices: 2
· product links: 4
· hreflang: 0
· canonical: https://www2.hm.com/en_us/index.html
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 1609153 | 3991 | — | · | 5 | - |
| Chrome (DC) | 403 | 543 | 313 | 1% | · | 0 | http_403 |
| Googlebot | 403 | 379 | 299 | 1% | · | 0 | http_403 |
| Bingbot | 403 | 379 | 299 | 1% | · | 0 | http_403 |
| GPTBot | 403 | 379 | 299 | 1% | · | 0 | http_403 |
| ChatGPT-User | 0 | 0 | — | — | · | 0 | timeout |
| ClaudeBot | 403 | 379 | 299 | 1% | · | 0 | http_403 |
| PerplexityBot | 0 | 0 | — | — | · | 0 | timeout |
#68 uniqlo.com fashion · Global/JP AI-blocked · AI avg: 0.0/5
title:
Women's Clothing & Accessories | UNIQLO US
visible text: 9652 chars · JSON-LD types:
—
· prices: 16
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 554067 | 9652 | — | · | 4 | - |
| Chrome (DC) | 200 | 1648064 | 1563 | 36% | · | 4 | - |
| Googlebot | 200 | 1646593 | 1563 | 36% | · | 4 | - |
| Bingbot | 0 | 0 | — | — | · | 0 | timeout |
| GPTBot | 0 | 0 | — | — | · | 0 | timeout |
| ChatGPT-User | 0 | 0 | — | — | · | 0 | timeout |
| ClaudeBot | 0 | 0 | — | — | · | 0 | timeout |
| PerplexityBot | 0 | 0 | — | — | · | 0 | timeout |
#69 lululemon.com fashion · US AI-blocked · AI avg: 0.0/5
title:
technical apparel + athletic shoes | lululemon
visible text: 6121 chars · JSON-LD types:
Organization, SearchAction, WebSite
· prices: 15
· product links: 45
· hreflang: 49
· canonical: https://shop.lululemon.com/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 1437598 | 6121 | — | · | 5 | - |
| Chrome (DC) | 200 | 1381829 | 6369 | 99% | · | 5 | - |
| Googlebot | 0 | 0 | — | — | · | 0 | timeout |
| Bingbot | 0 | 0 | — | — | · | 0 | timeout |
| GPTBot | 0 | 0 | — | — | · | 0 | timeout |
| ChatGPT-User | 0 | 0 | — | — | · | 0 | timeout |
| ClaudeBot | 0 | 0 | — | — | · | 0 | timeout |
| PerplexityBot | 0 | 0 | — | — | · | 0 | timeout |
#70 newegg.com electronics · US AI-ready · AI avg: 5.0/5
title:
Electronics Store: Tech, PC Parts, AI PC & More | Newegg
visible text: 88195 chars · JSON-LD types:
ContactPoint, Organization, WebSite
· prices: 79
· product links: 196
· hreflang: 0
· canonical: https://www.newegg.com/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 1460183 | 88195 | — | · | 5 | - |
| Chrome (DC) | 200 | 656021 | 10605 | 29% | · | 5 | - |
| Googlebot | 200 | 287235 | 4463 | 15% | · | 4 | - |
| Bingbot | 403 | 562587 | 626 | 1% | · | 0 | http_403 |
| GPTBot | 200 | 656021 | 10605 | 29% | · | 5 | - |
| ChatGPT-User | 200 | 656021 | 10605 | 29% | · | 5 | - |
| ClaudeBot | 200 | 656021 | 10605 | 29% | · | 5 | - |
| PerplexityBot | 200 | 656021 | 10605 | 29% | · | 5 | - |
#71 bhphotovideo.com electronics · US AI-blocked IP-walled · AI avg: 0.0/5
title:
B&H Photo Video Digital Cameras, Photography, Computers
visible text: 4256 chars · JSON-LD types:
Organization, SearchAction
· prices: 0
· product links: 0
· hreflang: 0
· canonical: https://www.bhphotovideo.com/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 203652 | 4256 | — | · | 4 | - |
| Chrome (DC) | 403 | 5859 | 58 | 0% | · | 0 | http_403 |
| Googlebot | 403 | 5795 | 58 | 0% | · | 0 | http_403 |
| Bingbot | 403 | 5646 | 58 | 0% | · | 0 | http_403 |
| GPTBot | 403 | 5518 | 58 | 0% | · | 0 | http_403 |
| ChatGPT-User | 403 | 5539 | 58 | 0% | · | 0 | http_403 |
| ClaudeBot | 403 | 5539 | 58 | 0% | · | 0 | http_403 |
| PerplexityBot | 403 | 5560 | 58 | 0% | · | 0 | http_403 |
#72 wish.com marketplace · Global partial · AI avg: 3.0/5
title:
Wish | Shop and Save
visible text: 2729 chars · JSON-LD types:
SearchAction, WebSite
· prices: 25
· product links: 14
· hreflang: 16
· canonical: https://www.wish.com/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 340853 | 2729 | — | · | 5 | - |
| Chrome (DC) | 200 | 194471 | 20 | 1% | · | 3 | - |
| Googlebot | 200 | 207092 | 20 | 1% | · | 3 | - |
| Bingbot | 403 | 4550 | 772 | 3% | · | 0 | http_403 |
| GPTBot | 200 | 198321 | 20 | 1% | · | 3 | - |
| ChatGPT-User | 200 | 206549 | 20 | 1% | · | 3 | - |
| ClaudeBot | 200 | 206544 | 20 | 1% | · | 3 | - |
| PerplexityBot | 200 | 202152 | 20 | 1% | · | 3 | - |
#73 stockx.com resale · US AI-blocked IP-walled · AI avg: 0.0/5
title:
StockX: Sneakers, Streetwear, Trading Cards, Handbags, Watches
visible text: 797 chars · JSON-LD types:
EntryPoint, MobileApplication, Offer, Organization, Person, Place, PostalAddress, SearchAction, WebSite
· prices: 7
· product links: 4
· hreflang: 13
· canonical: https://stockx.com
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 519025 | 797 | — | · | 5 | - |
| Chrome (DC) | 403 | 5827 | 58 | 0% | · | 0 | http_403 |
| Googlebot | 403 | 5742 | 58 | 0% | · | 0 | http_403 |
| Bingbot | 403 | 5636 | 58 | 0% | · | 0 | http_403 |
| GPTBot | 403 | 5486 | 58 | 0% | · | 0 | http_403 |
| ChatGPT-User | 403 | 5508 | 58 | 0% | · | 0 | http_403 |
| ClaudeBot | 403 | 5486 | 58 | 0% | · | 0 | http_403 |
| PerplexityBot | 403 | 5529 | 58 | 0% | · | 0 | http_403 |
#74 jd.com marketplace · CN partial · AI avg: 3.0/5
title:
JD.com, Inc.
visible text: 15263 chars · JSON-LD types:
—
· prices: 1
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 76164 | 15263 | — | · | 4 | - |
| Chrome (DC) | 200 | 14708 | 579 | 0% | · | 3 | - |
| Googlebot | 200 | 14708 | 579 | 0% | · | 3 | - |
| Bingbot | 200 | 14708 | 579 | 0% | · | 3 | - |
| GPTBot | 200 | 14708 | 579 | 0% | · | 3 | - |
| ChatGPT-User | 200 | 14708 | 579 | 0% | · | 3 | - |
| ClaudeBot | 200 | 14708 | 579 | 0% | · | 3 | - |
| PerplexityBot | 200 | 14708 | 579 | 0% | · | 3 | - |
#75 pinduoduo.com marketplace · CN partial · AI avg: 3.0/5
title:
拼多多 新电商开创者
visible text: 611 chars · JSON-LD types:
—
· prices: 0
· product links: 0
· hreflang: 2
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 30694 | 611 | — | · | 3 | - |
| Chrome (DC) | 200 | 11919 | 531 | 82% | · | 3 | - |
| Googlebot | 200 | 11919 | 531 | 82% | · | 3 | - |
| Bingbot | 200 | 11919 | 531 | 82% | · | 3 | - |
| GPTBot | 200 | 11919 | 531 | 82% | · | 3 | - |
| ChatGPT-User | 200 | 11919 | 531 | 82% | · | 3 | - |
| ClaudeBot | 200 | 11919 | 531 | 82% | · | 3 | - |
| PerplexityBot | 200 | 11919 | 531 | 82% | · | 3 | - |
#76 1688.com marketplace-b2b · CN thin · AI avg: 1.0/5
title:
visible text: 0 chars · JSON-LD types:
—
· prices: 0
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 226282 | 0 | — | · | 1 | - |
| Chrome (DC) | 200 | 93050 | 0 | — | ✓ | 1 | - |
| Googlebot | 200 | 1729 | 0 | — | ✓ | 0 | captcha_page |
| Bingbot | 200 | 93050 | 0 | — | ✓ | 1 | - |
| GPTBot | 200 | 93050 | 0 | — | ✓ | 1 | - |
| ChatGPT-User | 200 | 93050 | 0 | — | ✓ | 1 | - |
| ClaudeBot | 200 | 93050 | 0 | — | ✓ | 1 | - |
| PerplexityBot | 200 | 93055 | 0 | — | ✓ | 1 | - |
#77 tmall.com marketplace · CN partial · AI avg: 3.0/5
title:
天猫Tmall.com - 买正品上天猫就购了
visible text: 891 chars · JSON-LD types:
—
· prices: 0
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 386951 | 891 | — | · | 3 | - |
| Chrome (DC) | 200 | 108568 | 708 | 100% | · | 3 | - |
| Googlebot | 200 | 3508 | 7 | 12% | · | 2 | - |
| Bingbot | 200 | 108568 | 708 | 100% | · | 3 | - |
| GPTBot | 200 | 108568 | 708 | 100% | · | 3 | - |
| ChatGPT-User | 200 | 108568 | 708 | 100% | · | 3 | - |
| ClaudeBot | 200 | 108568 | 708 | 100% | · | 3 | - |
| PerplexityBot | 200 | 108568 | 708 | 100% | · | 3 | - |
#78 lazada.com marketplace · SEA partial · AI avg: 3.0/5
title:
Home
visible text: 1731 chars · JSON-LD types:
—
· prices: 0
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 178265 | 1731 | — | · | 3 | - |
| Chrome (DC) | 200 | 27329 | 1695 | 96% | · | 3 | - |
| Googlebot | 200 | 27329 | 1695 | 96% | · | 3 | - |
| Bingbot | 200 | 27329 | 1695 | 96% | · | 3 | - |
| GPTBot | 200 | 27329 | 1695 | 96% | · | 3 | - |
| ChatGPT-User | 200 | 27329 | 1695 | 96% | · | 3 | - |
| ClaudeBot | 200 | 27329 | 1695 | 96% | · | 3 | - |
| PerplexityBot | 200 | 27329 | 1695 | 96% | · | 3 | - |
#79 shopee.sg marketplace · SG AI-blocked · AI avg: 0.2/5
title:
visible text: 0 chars · JSON-LD types:
—
· prices: 0
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 159702 | 0 | — | · | 1 | - |
| Chrome (DC) | 200 | 142129 | 0 | — | ✓ | 1 | - |
| Googlebot | 403 | 129 | 128 | — | · | 0 | http_403 |
| Bingbot | 403 | 130 | 129 | — | · | 0 | http_403 |
| GPTBot | 403 | 130 | 129 | — | · | 0 | http_403 |
| ChatGPT-User | 403 | 129 | 128 | — | · | 0 | http_403 |
| ClaudeBot | 200 | 156516 | 41 | — | · | 1 | - |
| PerplexityBot | 403 | 130 | 129 | — | · | 0 | http_403 |
#80 gmarket.co.kr marketplace · KR AI-blocked IP-walled · AI avg: 0.0/5
title:
G마켓 - 지금부터의 마켓
visible text: 24758 chars · JSON-LD types:
—
· prices: 0
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 1203548 | 24758 | — | · | 3 | - |
| Chrome (DC) | 403 | 20979 | 271 | 0% | · | 0 | http_403 |
| Googlebot | 403 | 20936 | 271 | 0% | · | 0 | http_403 |
| Bingbot | 403 | 20808 | 271 | 0% | · | 0 | http_403 |
| GPTBot | 403 | 20680 | 271 | 0% | · | 0 | http_403 |
| ChatGPT-User | 403 | 20659 | 271 | 0% | · | 0 | http_403 |
| ClaudeBot | 403 | 20680 | 271 | 0% | · | 0 | http_403 |
| PerplexityBot | 403 | 20701 | 271 | 0% | · | 0 | http_403 |
#81 carrefour.fr grocery · FR AI-blocked IP-walled · AI avg: 0.0/5
title:
Carrefour : Magasins et Courses en ligne (Drive, Livraison à Domicile)
visible text: 9953 chars · JSON-LD types:
Organization, SearchAction, WebSite
· prices: 25
· product links: 55
· hreflang: 0
· canonical: https://www.carrefour.fr/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 410151 | 9953 | — | · | 5 | - |
| Chrome (DC) | 403 | 573967 | 205 | 1% | · | 0 | http_403 |
| Googlebot | 403 | 573882 | 205 | 1% | · | 0 | http_403 |
| Bingbot | 403 | 573754 | 205 | 1% | · | 0 | http_403 |
| GPTBot | 403 | 573626 | 205 | 1% | · | 0 | http_403 |
| ChatGPT-User | 403 | 573626 | 205 | 1% | · | 0 | http_403 |
| ClaudeBot | 403 | 573626 | 205 | 1% | · | 0 | http_403 |
| PerplexityBot | 403 | 573647 | 205 | 1% | · | 0 | http_403 |
#82 johnlewis.com department · UK AI-blocked IP-walled · AI avg: 0.0/5
title:
John Lewis & Partners | Never Knowingly Undersold
visible text: 6714 chars · JSON-LD types:
ContactPoint, Organization, SearchAction, WebSite
· prices: 12
· product links: 0
· hreflang: 0
· canonical: https://www.johnlewis.com/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 599036 | 6714 | — | · | 5 | - |
| Chrome (DC) | 0 | 0 | — | — | · | 0 | timeout |
| Googlebot | 0 | 0 | — | — | · | 0 | timeout |
| Bingbot | 0 | 0 | — | — | · | 0 | timeout |
| GPTBot | 0 | 0 | — | — | · | 0 | timeout |
| ChatGPT-User | 0 | 0 | — | — | · | 0 | timeout |
| ClaudeBot | 0 | 0 | — | — | · | 0 | timeout |
| PerplexityBot | 0 | 0 | — | — | · | 0 | timeout |
#83 argos.co.uk department · UK AI-blocked · AI avg: 0.0/5
title:
Arrow down
visible text: 29541 chars · JSON-LD types:
ContactPoint, ImageObject, MerchantReturnPolicy, OnlineStore, WebSite
· prices: 35
· product links: 44
· hreflang: 0
· canonical: https://www.argos.co.uk
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 1557055 | 29541 | — | · | 5 | - |
| Chrome (DC) | 200 | 709929 | 26045 | 92% | · | 5 | - |
| Googlebot | 403 | 371 | 291 | 0% | · | 0 | http_403 |
| Bingbot | 403 | 371 | 291 | 0% | · | 0 | http_403 |
| GPTBot | 403 | 373 | 293 | 0% | · | 0 | http_403 |
| ChatGPT-User | 403 | 371 | 291 | 0% | · | 0 | http_403 |
| ClaudeBot | 403 | 371 | 291 | 0% | · | 0 | http_403 |
| PerplexityBot | 403 | 371 | 291 | 0% | · | 0 | http_403 |
#84 currys.co.uk electronics · UK AI-blocked IP-walled · AI avg: 0.0/5
title:
Currys | Washing Machines, Laptops, TVs, Consoles
visible text: 13691 chars · JSON-LD types:
Organization
· prices: 41
· product links: 21
· hreflang: 2
· canonical: https://www.currys.co.uk/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 740529 | 13691 | — | · | 5 | - |
| Chrome (DC) | 403 | 5965 | 770 | 4% | · | 0 | http_403 |
| Googlebot | 403 | 5965 | 770 | 4% | · | 0 | http_403 |
| Bingbot | 403 | 5965 | 770 | 4% | · | 0 | http_403 |
| GPTBot | 403 | 5965 | 770 | 4% | · | 0 | http_403 |
| ChatGPT-User | 403 | 5965 | 770 | 4% | · | 0 | http_403 |
| ClaudeBot | 403 | 5965 | 770 | 4% | · | 0 | http_403 |
| PerplexityBot | 403 | 5965 | 770 | 4% | · | 0 | http_403 |
#85 otto.de marketplace · DE AI-ready · AI avg: 5.0/5
title:
OTTO - Mode, Möbel & Technik » Zum Online-Shop
visible text: 16419 chars · JSON-LD types:
Organization
· prices: 10
· product links: 92
· hreflang: 0
· canonical: https://www.otto.de/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 1082088 | 16419 | — | · | 5 | - |
| Chrome (DC) | 200 | 391200 | 13311 | 88% | · | 5 | - |
| Googlebot | 200 | 387666 | 13311 | 88% | · | 5 | - |
| Bingbot | 200 | 388812 | 13311 | 88% | · | 5 | - |
| GPTBot | 200 | 389106 | 13311 | 88% | · | 5 | - |
| ChatGPT-User | 200 | 389684 | 13311 | 88% | · | 5 | - |
| ClaudeBot | 200 | 399828 | 13311 | 88% | · | 5 | - |
| PerplexityBot | 200 | 390388 | 13311 | 88% | · | 5 | - |
#86 bol.com marketplace · NL unreachable · AI avg: 0.0/5
title:
visible text: 32 chars · JSON-LD types:
—
· prices: 0
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 19003 | 32 | — | · | 0 | captcha_page |
| Chrome (DC) | 200 | 556037 | 23662 | 0% | · | 4 | - |
| Googlebot | 403 | 9568 | 1160 | 0% | · | 0 | http_403 |
| Bingbot | 403 | 9568 | 1160 | 0% | · | 0 | http_403 |
| GPTBot | 403 | 9568 | 1160 | 0% | · | 0 | http_403 |
| ChatGPT-User | 403 | 9568 | 1160 | 0% | · | 0 | http_403 |
| ClaudeBot | 403 | 9568 | 1160 | 0% | · | 0 | http_403 |
| PerplexityBot | 403 | 9568 | 1160 | 0% | · | 0 | http_403 |
#87 cdiscount.com marketplace · FR partial · AI avg: 3.0/5
title:
Cdiscount : des prix bas qui ont de la voix !
visible text: 22324 chars · JSON-LD types:
SearchAction, WebSite
· prices: 3
· product links: 0
· hreflang: 0
· canonical: https://www.cdiscount.com/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 527517 | 22324 | — | · | 5 | - |
| Chrome (DC) | 200 | 14175 | 297 | 1% | · | 2 | - |
| Googlebot | 200 | 14282 | 297 | 1% | · | 2 | - |
| Bingbot | 403 | 13204 | 493 | 1% | · | 0 | http_403 |
| GPTBot | 200 | 14154 | 297 | 1% | · | 2 | - |
| ChatGPT-User | 200 | 424893 | 13239 | 67% | · | 4 | - |
| ClaudeBot | 200 | 14154 | 297 | 1% | · | 2 | - |
| PerplexityBot | 200 | 424893 | 13239 | 67% | · | 4 | - |
#88 fnac.com books-electronics · FR AI-blocked IP-walled · AI avg: 0.0/5
title:
Fnac.com : acheter, choisir, comparer en ligne tous les produits culturels et high tech de l'actualité
visible text: 285 chars · JSON-LD types:
—
· prices: 0
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 190753 | 285 | — | · | 2 | - |
| Chrome (DC) | 503 | 173941 | 285 | 100% | · | 0 | http_503 |
| Googlebot | 403 | 65479 | 86 | 4% | · | 0 | http_403 |
| Bingbot | 403 | 70173 | 86 | 4% | · | 0 | http_403 |
| GPTBot | 403 | 65479 | 86 | 4% | · | 0 | http_403 |
| ChatGPT-User | 403 | 70174 | 86 | 4% | · | 0 | http_403 |
| ClaudeBot | 403 | 65479 | 86 | 4% | · | 0 | http_403 |
| PerplexityBot | 403 | 65479 | 86 | 4% | · | 0 | http_403 |
#89 mediamarkt.de electronics · DE AI-blocked IP-walled · AI avg: 0.0/5
title:
Elektronik, Trends & Technik kaufen im Onlineshop | MediaMarkt
visible text: 29130 chars · JSON-LD types:
ContactPoint, Organization, PostalAddress, SearchAction, WebSite
· prices: 14
· product links: 8
· hreflang: 16
· canonical: https://www.mediamarkt.de/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 1006556 | 29130 | — | · | 5 | - |
| Chrome (DC) | 403 | 13522 | 210 | 0% | · | 0 | http_403 |
| Googlebot | 403 | 13458 | 210 | 0% | · | 0 | http_403 |
| Bingbot | 403 | 13330 | 210 | 0% | · | 0 | http_403 |
| GPTBot | 403 | 13202 | 210 | 0% | · | 0 | http_403 |
| ChatGPT-User | 403 | 13202 | 210 | 0% | · | 0 | http_403 |
| ClaudeBot | 403 | 13202 | 210 | 0% | · | 0 | http_403 |
| PerplexityBot | 403 | 13223 | 210 | 0% | · | 0 | http_403 |
#90 saturn.de electronics · DE AI-blocked IP-walled · AI avg: 0.0/5
title:
Elektronik, Technik und Trends im Onlineshop | Saturn
visible text: 28373 chars · JSON-LD types:
ContactPoint, Organization, PostalAddress, SearchAction, WebSite
· prices: 14
· product links: 8
· hreflang: 1
· canonical: https://www.saturn.de
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 944835 | 28373 | — | · | 5 | - |
| Chrome (DC) | 403 | 7794 | 202 | 1% | · | 0 | http_403 |
| Googlebot | 403 | 7708 | 202 | 1% | · | 0 | http_403 |
| Bingbot | 403 | 7602 | 202 | 1% | · | 0 | http_403 |
| GPTBot | 403 | 7474 | 202 | 1% | · | 0 | http_403 |
| ChatGPT-User | 403 | 7474 | 202 | 1% | · | 0 | http_403 |
| ClaudeBot | 403 | 7474 | 202 | 1% | · | 0 | http_403 |
| PerplexityBot | 403 | 7495 | 202 | 1% | · | 0 | http_403 |
#91 jumia.com.ng marketplace · AF unreachable · AI avg: 0.0/5
title:
visible text: — chars · JSON-LD types:
—
· prices: 0
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 0 | 0 | — | — | · | 0 | network_error |
| Chrome (DC) | 403 | 5923 | 58 | — | · | 0 | http_403 |
| Googlebot | 403 | 5816 | 58 | — | · | 0 | http_403 |
| Bingbot | 403 | 5688 | 58 | — | · | 0 | http_403 |
| GPTBot | 403 | 5560 | 58 | — | · | 0 | http_403 |
| ChatGPT-User | 403 | 5560 | 58 | — | · | 0 | http_403 |
| ClaudeBot | 403 | 5560 | 58 | — | · | 0 | http_403 |
| PerplexityBot | 403 | 5582 | 58 | — | · | 0 | http_403 |
#92 decathlon.com sports · Global/FR AI-ready · AI avg: 5.0/5
title:
Decathlon America | Outdoor Sports Clothing & Gear
visible text: 9272 chars · JSON-LD types:
Organization
· prices: 49
· product links: 65
· hreflang: 0
· canonical: https://www.decathlon.com/
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 912863 | 9272 | — | · | 5 | - |
| Chrome (DC) | 200 | 894971 | 7316 | 65% | · | 5 | - |
| Googlebot | 200 | 894841 | 7316 | 65% | · | 5 | - |
| Bingbot | 200 | 894841 | 7316 | 65% | · | 5 | - |
| GPTBot | 200 | 894971 | 7316 | 65% | · | 5 | - |
| ChatGPT-User | 200 | 894841 | 7316 | 65% | · | 5 | - |
| ClaudeBot | 200 | 894971 | 7316 | 65% | · | 5 | - |
| PerplexityBot | 200 | 894971 | 7316 | 65% | · | 5 | - |
#93 canadiantire.ca big-box · CA AI-blocked · AI avg: 3.8/5
title:
Shop Canada’s Top Department Store Online & at 500+ Locations | Canadian Tire
visible text: 37249 chars · JSON-LD types:
Action, ContactPoint, Corporation, EntryPoint, ItemList, PostalAddress, SearchAction, SiteNavigationElement, WPFooter, WPHeader, WebSite
· prices: 64
· product links: 2
· hreflang: 3
· canonical: https://www.canadiantire.ca/en.html
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 703295 | 37249 | — | · | 5 | - |
| Chrome (DC) | 200 | 395658 | 56005 | 45% | · | 5 | - |
| Googlebot | 0 | 0 | — | — | · | 0 | timeout |
| Bingbot | 0 | 0 | — | — | · | 0 | timeout |
| GPTBot | 200 | 4166980 | 350133 | 12% | · | 5 | - |
| ChatGPT-User | 200 | 4166980 | 350133 | 12% | · | 5 | - |
| ClaudeBot | 0 | 0 | — | — | · | 0 | timeout |
| PerplexityBot | 200 | 4166980 | 350133 | 12% | · | 5 | - |
#94 ao.com electronics · UK unreachable · AI avg: 0.0/5
title:
visible text: — chars · JSON-LD types:
—
· prices: 0
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 0 | 0 | — | — | · | 0 | network_error |
| Chrome (DC) | 403 | 5823 | 58 | — | · | 0 | http_403 |
| Googlebot | 403 | 5759 | 58 | — | · | 0 | http_403 |
| Bingbot | 403 | 5632 | 58 | — | · | 0 | http_403 |
| GPTBot | 403 | 5503 | 58 | — | · | 0 | http_403 |
| ChatGPT-User | 403 | 5503 | 58 | — | · | 0 | http_403 |
| ClaudeBot | 403 | 5503 | 58 | — | · | 0 | http_403 |
| PerplexityBot | 403 | 5524 | 58 | — | · | 0 | http_403 |
#95 very.co.uk department · UK unreachable · AI avg: 0.0/5
title:
Access Denied
visible text: 574 chars · JSON-LD types:
—
· prices: 0
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 21144 | 574 | — | · | 0 | captcha_page |
| Chrome (DC) | 403 | 4462 | 590 | 81% | · | 0 | http_403 |
| Googlebot | 403 | 4307 | 590 | 81% | · | 0 | http_403 |
| Bingbot | 403 | 4307 | 590 | 81% | · | 0 | http_403 |
| GPTBot | 403 | 4307 | 590 | 81% | · | 0 | http_403 |
| ChatGPT-User | 403 | 4307 | 590 | 81% | · | 0 | http_403 |
| ClaudeBot | 403 | 4307 | 590 | 81% | · | 0 | http_403 |
| PerplexityBot | 403 | 4307 | 590 | 81% | · | 0 | http_403 |
#96 boots.com beauty · UK unreachable · AI avg: 0.0/5
title:
visible text: — chars · JSON-LD types:
—
· prices: 0
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 0 | 0 | — | — | · | 0 | network_error |
| Chrome (DC) | 200 | 1159 | 82 | — | · | 1 | - |
| Googlebot | 403 | 962 | 82 | — | · | 0 | http_403 |
| Bingbot | 403 | 962 | 82 | — | · | 0 | http_403 |
| GPTBot | 403 | 962 | 82 | — | · | 0 | http_403 |
| ChatGPT-User | 403 | 962 | 82 | — | · | 0 | http_403 |
| ClaudeBot | 403 | 962 | 82 | — | · | 0 | http_403 |
| PerplexityBot | 403 | 962 | 82 | — | · | 0 | http_403 |
#97 catch.com.au marketplace · AU unreachable · AI avg: 0.0/5
title:
Catch - Maintenance
visible text: 181 chars · JSON-LD types:
—
· prices: 0
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 22288 | 181 | — | · | 0 | captcha_page |
| Chrome (DC) | 403 | 5465 | 181 | 100% | ✓ | 0 | http_403 |
| Googlebot | 403 | 5465 | 181 | 100% | ✓ | 0 | http_403 |
| Bingbot | 403 | 5465 | 181 | 100% | ✓ | 0 | http_403 |
| GPTBot | 403 | 5465 | 181 | 100% | ✓ | 0 | http_403 |
| ChatGPT-User | 403 | 5465 | 181 | 100% | ✓ | 0 | http_403 |
| ClaudeBot | 403 | 5465 | 181 | 100% | ✓ | 0 | http_403 |
| PerplexityBot | 403 | 5465 | 181 | 100% | ✓ | 0 | http_403 |
#98 iherb.com health · Global/US AI-blocked IP-walled · AI avg: 0.0/5
title:
iHerb | Vitamins, Supplements, Natural Health Products
visible text: 42050 chars · JSON-LD types:
Organization
· prices: 217
· product links: 0
· hreflang: 101
· canonical: https://www.iherb.com
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 3478390 | 42050 | — | · | 5 | - |
| Chrome (DC) | 403 | 5831 | 58 | 0% | · | 0 | http_403 |
| Googlebot | 403 | 5745 | 58 | 0% | · | 0 | http_403 |
| Bingbot | 403 | 5617 | 58 | 0% | · | 0 | http_403 |
| GPTBot | 403 | 5511 | 58 | 0% | · | 0 | http_403 |
| ChatGPT-User | 403 | 5511 | 58 | 0% | · | 0 | http_403 |
| ClaudeBot | 403 | 5511 | 58 | 0% | · | 0 | http_403 |
| PerplexityBot | 403 | 5532 | 58 | 0% | · | 0 | http_403 |
#99 bunnings.com.au home · AU unreachable · AI avg: 0.0/5
title:
visible text: — chars · JSON-LD types:
—
· prices: 0
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 0 | 0 | — | — | · | 0 | network_error |
| Chrome (DC) | 403 | 5943 | 58 | — | · | 0 | http_403 |
| Googlebot | 403 | 5879 | 58 | — | · | 0 | http_403 |
| Bingbot | 403 | 5730 | 58 | — | · | 0 | http_403 |
| GPTBot | 403 | 5602 | 58 | — | · | 0 | http_403 |
| ChatGPT-User | 403 | 5623 | 58 | — | · | 0 | http_403 |
| ClaudeBot | 403 | 5623 | 58 | — | · | 0 | http_403 |
| PerplexityBot | 403 | 5645 | 58 | — | · | 0 | http_403 |
#100 kogan.com marketplace · AU unreachable · AI avg: 1.2/5
title:
kogan.com
visible text: 9 chars · JSON-LD types:
—
· prices: 0
· product links: 0
· hreflang: 0
· canonical: —
| UA | HTTP | bytes | text | jaccard | ≡SHA | score | block |
|---|---|---|---|---|---|---|---|
| Chrome (resi) | 200 | 18139 | 9 | — | · | 0 | captcha_page |
| Chrome (DC) | 403 | 769 | 53 | 25% | · | 0 | http_403 |
| Googlebot | 403 | 769 | 53 | 25% | · | 0 | http_403 |
| Bingbot | 403 | 769 | 53 | 25% | · | 0 | http_403 |
| GPTBot | 200 | 1533279 | 40286 | 0% | · | 5 | - |
| ChatGPT-User | 403 | 769 | 53 | 25% | · | 0 | http_403 |
| ClaudeBot | 403 | 769 | 53 | 25% | · | 0 | http_403 |
| PerplexityBot | 403 | 769 | 53 | 25% | · | 0 | http_403 |
5 · Methodology
Methodology · click to expand
Per site, 8 fetches in parallel:
chrome_baseline— residential proxy via HasData with full JS rendering. This is the "real-user reference"; comparisons are made against this body.chrome_direct— bare HTTP from our Hetzner DC IP. Difference from baseline reveals whether the site IP-walls datacenter traffic regardless of UA.googlebot,bingbot,gptbot,chatgpt_user,claudebot,perplexitybot— bare HTTP, official declared UA, no JS execution. This mirrors how the real AI crawlers behave (Vercel 2025: none of the major AI crawlers run JS).
Per fetch we extract: HTTP status, byte length, <title>, <h1>, canonical, robots meta,
all JSON-LD @type values, currency tokens, visible price count, hreflang count,
product-pattern link count, SHA-256 of normalized visible text, and a word-token set used for Jaccard similarity.
AI-readiness score 0..5: +1 for status 200; +1 for >500 visible chars; +1 for a present title; +1 for any JSON-LD; +1 for Product/Offer schema, a visible price, or ≥5 product-shaped links.
Per-site verdict:
ai_ready = min AI-bot score ≥4 ·
partial = max AI-bot score 3-4 ·
thin = max AI-bot score ≤2 ·
ai_blocked = at least one AI bot hard-blocked ·
unreachable = even the residential-proxy baseline failed.
Retries: one re-fetch after 2 s on block · Concurrency: 6 sites in parallel, 16 direct curls in flight, 3 HasData calls in flight.
Raw HTML and per-site JSON for every cell: audit/2026-05-ecommerce-100/data/ on the host.