VitalSentinel bots and how to control them
Information about our monitoring bots, how to identify them and what they do when they visit your website.
Last updated: October 25, 2026
Why am I seeing VitalSentinel in my logs?
If you are seeing VitalSentinel requests in your server logs, someone has added your website to VitalSentinel or has run one of our free checks on it. Each bot below says how often it visits and how to allow or stop it. If you have concerns, please contact us.
VitalSentinel Bot
Site Health and Page Monitoring crawler, robots.txt and sitemap checks
One name, shared by several activities, so the log line alone will not tell you which one called. The crawler for Site Health and Page Monitoring fetches and renders pages. The robots.txt monitor reads /robots.txt, the sitemap scanner reads your sitemaps, and a few single requests check the homepage, for example to verify ownership.
This identity contacts a monitored website only once the site is verified in VitalSentinel, normally by its owner. The exceptions are the requests needed to add and verify a site, and a one-time transition period for sites added before verification was required. It never calls while a site's monitoring is paused or suspended. It also reads a few public files published by search engines and AI vendors, such as their crawler IP lists.
User agent
VitalSentinel Bot/1.2 (+https://www.vitalsentinel.com/bot)robots.txt token
VitalSentinel-BotRobots.txt matches on the product token, so VitalSentinel-Bot and VitalSentinel Bot both match. Only the crawler obeys it: see “How to allow or stop the crawler” below.
Request frequency
Site Health: about 1 page a minute per site (1 every 5 minutes on the Free plan), with short bursts of up to 3 a minute on paid plans. Page Monitoring: the pages a customer chose to watch, at most 60 page loads a minute per site. Each page load also fetches the files the page itself loads. The other checks run on a schedule or when a user asks for one.
Behavior
- Only the crawler obeys robots.txt, and nothing overrides it. The other activities send the same user agent and token but do not consult robots.txt
- The crawler renders pages in headless Chrome and loads the scripts, styles and API calls the page loads
- The crawler checks images and media without downloading them
- Backs off when your server answers 429 or sends Retry-After
- The crawler starts every visit with no cookies and no login, sends back only cookies your site sets during that visit, and never clicks a consent banner. Over https it normally signs its requests, as described under “Verifying the crawler” below
- The crawler blocks known analytics and advertising requests, so its visits are generally not counted by those tools
- Fetches only what an anonymous visitor would receive
VitalSentinel Uptime Bot
HTTP/HTTPS uptime and availability monitoring
Sends periodic HTTP requests to check that a website is online and responding correctly, and checks its SSL certificate.
User agent
VitalSentinel Uptime Bot/1.2 (+https://www.vitalsentinel.com/bot)robots.txt token
VitalSentinel-Uptime-BotThis is the token you would write in a User-agent: line. Please read “How to stop VitalSentinel traffic” below before you rely on it – a Disallow rule does not currently stop this fetch.
Request frequency
As often as every minute on paid plans, every 5 minutes on the Free plan
Behavior
- Requests the URLs a customer configured and follows redirects
- Records the status code and response time, and checks the SSL certificate
- Does not run JavaScript or load other resources
- Does not store page content
Synthetic Monitoring (Lighthouse)
Lighthouse performance audits and Core Web Vitals measurement
Synthetic tests send no VitalSentinel user agent. The default location runs on Google's PageSpeed Insights, so those requests come from Google with Google's user agent. The additional locations available from the Lite plan up run Lighthouse in headless Chrome on our own servers.
User agent
None specific to VitalSentinel – see the description above.
Request frequency
Every 24 hours, with 12-hour and 8-hour options on higher plans
Behavior
- Runs Lighthouse audits (Performance, Accessibility, Best practices, SEO) on mobile and desktop
- Measures Core Web Vitals and lab metrics (LCP, CLS, FCP, TBT, TTI, Speed Index)
- Captures screenshots of the page load
VitalSentinel Inspector Bot
The free page audit at vitalsentinel.com/tools/inspector
Not a crawler. It runs only when someone enters one URL into our free page audit. It loads that single page in headless Chrome, follows no links and does not run on a schedule. Whoever runs the audit does not have to own the site, as with any online checker.
User agent
Mozilla/5.0 (compatible; VitalSentinel Inspector Bot/1.2; +https://www.vitalsentinel.com/bot)Request frequency
On demand only – once per audit a person runs by hand
What we keep from your page
We keep the report, which includes the address audited, a final capture of the page and the frames recorded while it loaded. All of it is deleted automatically after 30 days. It can be opened only by someone who has the audit's unguessable link, and there is no index of audits.
Behavior
- Loads one URL in headless Chrome, plus a few supporting requests such as /robots.txt, the sitemaps it lists and a check that http:// redirects to https://
- Also has Google PageSpeed Insights test the page, so Google visits it with Google's own user agent
- Sends the user agent above on every request, including the browser load
- Follows no links and requests no other page of your site
- Fetches only what an anonymous visitor would receive
- Obeys the fair-use limits published on the tool page
VitalSentinel Free Tools Bot
The free tools at vitalsentinel.com/tools
Runs only when a person enters a URL into one of our free tools, such as a robots.txt test or a redirect trace. Nothing is scheduled. Whoever runs the check does not have to own the site, as with any online checker.
User agent
Mozilla/5.0 (compatible; VitalSentinel Free Tools Bot/1.0; +https://www.vitalsentinel.com/bot)Request frequency
On demand only – once per check a person runs by hand
What we keep from your page
One usage record per run is kept for 180 days. It holds the hostname checked, never the full address, and no IP address or page content. A result may be cached for up to an hour, so repeat checks do not reach your site again.
Behavior
- Fetches the submitted URL and only what that check needs, such as /robots.txt, the child sitemaps of a sitemap index or the hreflang alternates the page names
- Does not crawl or walk your site, and the requests per check are capped
- Sends the user agent above on every request, including browser loads
- Obeys the fair-use limits published on every tool page
- Fetches only what an anonymous visitor would receive
How to allow or stop the crawler
IP addresses
The crawler requests from these addresses.
194.163.166.126157.173.116.60
The same list is published as JSON, in the prefixes format Googlebot uses, at /.well-known/vitalsentinel-crawler-ips.json and, with the rest of these facts, at app.vitalsentinel.com/api/public/v1/crawler (no account or key needed). A user agent alone can be forged, so allowlist by address.
Verifying the crawler
Requests over https are normally signed with Web Bot Auth (HTTP Message Signatures, Ed25519) and carry Signature-Agent: "https://app.vitalsentinel.com". The public key is published in the key directory at https://app.vitalsentinel.com/.well-known/http-message-signatures-directory. Cloudflare has approved VitalSentinel Bot as a verified bot.
How to allow it
Allow the addresses above. Do not rate-limit them below 60 page loads a minute per site: a single page load fetches many files, and rate limits tuned for people can block a render halfway. Menu names change, so treat these steps as a guide.
- Any WAF or CDN. Allow the addresses above, and requests whose User-Agent contains VitalSentinel Bot. Exempt the addresses from rate limiting and bot scoring.
- Cloudflare. Security > WAF > Tools > IP Access Rules: add each address with the action Allow. If Bot Fight Mode or a rate-limiting rule still blocks it, add a WAF custom rule that skips them when the source IP is in the list.
- Akamai. Add the addresses to a Network List and exempt it in Bot Manager and the rate-control policy.
- Imperva. Add each address as an allowed IP in your security rules (an IP exception in the WAF and in bot protection) and exempt them from rate-limiting rules.
- AWS WAF. Create an IP set with the addresses and a rule with the action Allow, ordered before any rate-based rule and any managed bot-control rule group.
- Sucuri. Firewall > Access Control > Whitelist IP Addresses: add each address.
- Wordfence. Firewall > Allowlisted IPs: add each address (Wordfence > All Options).
How to stop it
The crawler always obeys robots.txt, and nothing overrides that. Add this to stop it. It applies to the crawler only:
User-agent: VitalSentinel-Bot Disallow: /
The crawler falls back to your User-agent: * group when it has none of its own, and honors a Crawl-delay for its token (or *) as the minimum gap between page loads. A 5xx or an unreachable robots.txt is treated as “disallow all”. You can also block the addresses at your firewall.
How to stop VitalSentinel traffic
Apart from the crawler, which has its own section above, our services do not consult a robots.txt Disallow rule, so adding one will not stop them.
To stop them, block the user agents listed above at your CDN, WAF or web server. Synthetic tests send no VitalSentinel user agent, so no user-agent rule will catch them. If you would rather we stopped checking your site from our side, email us.
One thing worth knowing before you block: these checks run because one of our customers asked us to watch your site. If you block them, whoever that is stops receiving results.
Questions, or want something removed?
If you have questions about our bots, or about a page of yours that we fetched, email support@vitalsentinel.com.
If we audited a page of yours, you can ask us what we hold about that audit and ask us to delete it. Send the report link, or the URL and the approximate date. Removing a report does not prevent someone from running a new audit.
For anything else, contact us.
This is what watching a site looks like
Right now these bots are checking other people's websites. The same checks can run for yours: Core Web Vitals from real users, indexing and robots.txt, then uptime and SSL, in one app.