NewVitalSentinel, redesigned
// Meta crawler

Meta-ExternalAgent:what it is and how to block it

Meta says Meta-ExternalAgent crawls the web for uses such as training foundation AI models and improving its products by indexing content directly.

robots.txt token
meta-externalagent
Operator
Meta
Purpose
AI training
Follows robots.txt
Says robots.txt controls access
User agent
Published
IP list
Not published
// user agent

Meta-ExternalAgent user agent

robots.txt token
meta-externalagent
Full user-agent string
meta-externalagent/1.1

Meta lists this string on its own or followed by a link to its crawler documentation. Meta writes the token in lowercase, and robots.txt matches tokens without regard to case.

Source: Meta for Developers: Meta web crawlers. Checked October 30, 2026.

// robots.txt

How to block Meta-ExternalAgent

Meta says to block it with a Disallow rule in robots.txt, and to allow up to 24 hours for a change because its crawlers may cache the file that long.

Blocking it stops future crawls by this token. Meta operates other crawlers for other purposes.

Meta-ExternalFetcher fetches links a user asked for and, Meta says, may bypass robots.txt. Meta-WebIndexer indexes pages for Meta AI search results. A Meta-ExternalAgent rule covers neither.

Block it from the whole site
User-agent: meta-externalagent
Disallow: /
Allow it everywhere
User-agent: meta-externalagent
Allow: /

Add the group to the robots.txt at the root of each host. A crawler with a group of its own ignores your User-agent: * rules, so repeat any you still want it to follow.

Write rules for all 20 AI crawlers at once
// verify

How to verify a real Meta-ExternalAgent visit

Meta publishes no IP ranges and no reverse DNS names for Meta-ExternalAgent, so a request that uses the name cannot be confirmed either way.

// your site

See whether Meta-ExternalAgent visits your site

robots.txt testerCheck whether your rules let Meta-ExternalAgent reach any path, and which line decides it.Open the toolAI crawler blockerSee what your live robots.txt allows Meta-ExternalAgent and the other 19 AI crawlers on our list today.Open the tool
Server Log MonitoringLists Meta-ExternalAgent by name with its hits, its 403s and whether it follows your robots.txt. Meta publishes no machine-readable address list, so its hits land in Cannot be checked.
Start for freeHow it works
// faq

Questions

Is Meta-ExternalAgent the same as facebookexternalhit?

No. facebookexternalhit builds the preview shown when someone shares your page on Facebook, Instagram or Messenger. It has its own token, so a Meta-ExternalAgent rule leaves those previews alone.

How long does a block take to work?

Meta says to allow up to 24 hours, because its crawlers may cache robots.txt for that long.

Does blocking Meta-ExternalAgent affect my Google rankings?

No. Meta-ExternalAgent is not Googlebot, so a rule for it changes nothing about how Google Search crawls, indexes or ranks your pages.

Other AI crawlers

// free to start

See every AI crawler that visits your site

Server Log Monitoring names every AI crawler in your server logs, with its hits and 403s, and flags hits that fake a crawler's name.

Start for freeSee pricing
1 domain freeNo credit cardAll 12 modules5.0on G2