Slackbot Link Unfurler
Fetches URLs posted in Slack messages to render the unfurl preview card inline in chat.
Specs
| Vendor | Slack |
| Category | |
| robots.txt token | Slackbot-LinkExpanding |
| Renders JavaScript | HTTP only |
| Honors robots.txt | no |
User-Agent string
Slackbot-LinkExpanding 1.0 (+https://api.slack.com/robots)Considerations
- Per Slack's own bot page: 'We do not currently honor robots.txt files' — Slack explicitly chose not to, because too much of the web is robots.txt-inaccessible to crawlers.
- Caches unfurl content for ~30 minutes.
- Block by returning the X-Slack-No-Retry: 1 header or 401/403 to the UA.
robots.txt recipe
# Block via response header in your app:
# return X-Slack-No-Retry: 1
# Or return 403 to UA matching Slackbot-LinkExpandingSources: Slack API · Link unfurling