PerplexityBot:what it is and how to block it
Perplexity says PerplexityBot surfaces and links websites in Perplexity's search results, and is not used to crawl content for AI foundation models.
- robots.txt token
PerplexityBot- Operator
- Perplexity
- Purpose
- AI search and assistant
- Follows robots.txt
- Says robots.txt controls access
- User agent
- Published
- IP list
- Published
PerplexityBot user agent
PerplexityBot
Mozilla/5.0 (compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)
An example from Perplexity, and the version number may change.
Source: Perplexity docs: Perplexity crawlers. Checked October 30, 2026.
How to block PerplexityBot
Perplexity lists PerplexityBot as a robots.txt token for managing access, and says a change can take up to 24 hours to apply.
Blocking it removes your site from Perplexity citations, which is where its referral traffic comes from.
Perplexity-User fetches a page when a person asks Perplexity something. Perplexity says it generally ignores robots.txt, because a user requested the fetch.
User-agent: PerplexityBot Disallow: /
User-agent: PerplexityBot Allow: /
Add the group to the robots.txt at the root of each host. A crawler with a group of its own ignores your User-agent: * rules, so repeat any you still want it to follow.
How to verify a real PerplexityBot visit
Perplexity publishes the crawler's IP ranges as JSON, and recommends matching the user agent and those ranges together, for example in a firewall rule.
https://www.perplexity.ai/perplexitybot.json
See whether PerplexityBot visits your site
Questions
Does PerplexityBot train AI models?
Perplexity says no: PerplexityBot is not used to crawl content for AI foundation models.
Will blocking PerplexityBot stop every Perplexity visit?
No. Perplexity-User fetches pages for user questions and, Perplexity says, generally ignores robots.txt rules.
Does blocking PerplexityBot affect my Google rankings?
No. PerplexityBot is not Googlebot, so a rule for it changes nothing about how Google Search crawls, indexes or ranks your pages.
Other AI crawlers
- ClaudeBotAnthropicCrawls pages for Anthropic's model training.Follows robots.txt, per its vendor
- GPTBotOpenAICrawls pages that may be used to train OpenAI models.Vendor says robots.txt controls access
- OAI-SearchBotOpenAIBuilds the index behind ChatGPT search results.Vendor says robots.txt controls access
- BytespiderByteDanceByteDance's web crawler. ByteDance publishes no documentation we could verify on what it crawls for.robots.txt behavior not stated
- Meta-ExternalAgentMetaCrawls pages Meta may use to train AI models and improve its products.Vendor says robots.txt controls access
- CCBotCommon CrawlBuilds the Common Crawl public web archive.Follows robots.txt, per its vendor
See every AI crawler that visits your site
Server Log Monitoring names every AI crawler in your server logs, with its hits and 403s, and flags hits that fake a crawler's name.