facebookexternalhit
| Vendor | Meta |
| Type | Social link-preview bot (Facebook, Instagram, Messenger, WhatsApp) |
| robots.txt token | facebookexternalhit |
| JavaScript rendering | No |
| Honors robots.txt | Partial — fetches link-preview metadata even when robots.txt disallows |
| Vendor docs | developers.facebook.com/docs/sharing/webmasters |
User-Agent string
facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)Purpose
The bot that fetches a URL whenever someone shares it on Facebook, Instagram, Messenger, or WhatsApp — to extract og:title, og:image, og:description and render the preview card. Older than the LLM era; this bot has been around since ~2010.
Quirks
- Fetches only the HTML head and a small body window — not designed
to crawl deep.
- Will fetch even if robots.txt disallows, on the grounds that the
fetch is on behalf of an end-user who shared the link. Meta's policy is that link-preview fetches are user actions, not crawls.
- For sites that want clean link previews,
og:andtwitter:
meta tags should be in static HTML — many SPAs that don't SSR these end up with broken FB previews.
How to control
# Provide og:* tags in static HTML for working previews:
<meta property="og:title" content="...">
<meta property="og:image" content="...">
<meta property="og:description" content="...">To block link previews:
User-agent: facebookexternalhit
Disallow: /(Note: Meta may not honor this for user-shared links.)