VitalSentinel Bots
Information about our monitoring bots, how to identify them, and what they do when visiting your website.
Why am I seeing VitalSentinel in my logs?
If you're seeing VitalSentinel requests in your server logs, it means your website is being monitored by one of our users. Our bots make a small number of requests and are designed to add negligible load to your server. If you have concerns, please contact us.
VitalSentinel Page Inspector (in-app)
On-demand audit of a single page, run by a customer from their dashboard
The inspector is not a crawler. It runs only when one of our customers asks us to audit one specific URL, it loads that single page in headless Chrome, it follows no links to any other page, and it does not run on a schedule. The URL is chosen by the customer and is not necessarily a site they own.
User Agent
VitalSentinel Inspector Bot/1.1 (+https://www.vitalsentinel.com/bot)robots.txt token
VitalSentinel-InspectorThis is the token you would write in a User-agent: line. Please read “How to stop VitalSentinel traffic” below before you rely on it – a Disallow rule does not currently stop this fetch.
Request Frequency
On demand only – never scheduled
What we keep from your page
We keep a copy of what we received: the rendered DOM, a HAR record of the page's network requests, a final screenshot and filmstrip frames. The DOM and network record are deleted automatically 12 months after the audit. Screenshots and filmstrip frames are stored separately and are deleted after 90 days, or sooner if the audit they belong to is deleted.
Behavior
- Loads one URL in headless Chrome and audits it
- The page load itself carries an ordinary Chrome user agent, with no VitalSentinel string in it. We would rather tell you that than let you search your logs for something that is not there. Sites that serve reduced markup to non-browser clients would otherwise hand the audit a different page than a real visitor gets, and the measurement would be wrong
- The requests around that page load do identify us: the plain HTTP fetch of the page, the robots.txt and sitemap reads, the stylesheet fetches and the HTTP/3 probe all send the one user agent above. That holds wherever the request originates - some of an audit's supporting fetches are performed by our uptime and robots.txt services on the Inspector's behalf, and they announce the Inspector, so one audit is one name in your logs rather than three
- Fetches the page exactly as any anonymous visitor would receive it – we do not log in, and we see nothing that is not publicly served
- Follows no links and requests no other page of your site
- Reads the page's meta robots tag and X-Robots-Tag header and reports them back to the customer
VitalSentinel Page Inspector (free tool)
The free page audit at vitalsentinel.com/tools/inspector
The inspector is not a crawler. It runs only when someone pastes one specific URL into our free page audit and presses a button. It loads that single page in headless Chrome, it follows no links to any other page, and it does not run on a schedule. Whoever ran the audit does not have to own the site, in the same way that anyone can paste any URL into any online checker.
User Agent
Mozilla/5.0 (compatible; VitalSentinel Inspector Bot/1.2; +https://www.vitalsentinel.com/bot)robots.txt token
VitalSentinel-InspectorThis is the token you would write in a User-agent: line. Please read “How to stop VitalSentinel traffic” below before you rely on it – a Disallow rule does not currently stop this fetch.
Request Frequency
On demand only – once per audit a person runs by hand
What we keep from your page
We keep a copy of what we received: the report itself, which includes the address audited, a final capture of the page and the filmstrip frames from the load. All of it is deleted automatically 30 days after the audit. It is reachable only at the audit's own link, which carries 128 bits of randomness and is the only access control; there is no index of audits anywhere, and no email address is collected.
Behavior
- Loads one URL in headless Chrome and audits it
- Sends the user agent above whether the request is the plain HTTP fetch of the page or the page load in a real browser, so one audit is one name in your logs rather than two
- The supporting requests around that page load send it too: the robots.txt and sitemap reads, and the certificate and transport probes
- Fetches the page exactly as any anonymous visitor would receive it – we do not log in, and we see nothing that is not publicly served
- Follows no links and requests no other page of your site
- Reads the page's meta robots tag and X-Robots-Tag header and reports them back on the audit
- Obeys the fair-use limits published on the tool page: a per-minute, per-hour and per-day cap per IP address – counted per /64 network prefix for IPv6 callers – plus a per-site cap
VitalSentinel Uptime Bot
HTTP/HTTPS uptime and availability monitoring
This bot performs periodic HTTP GET requests to check if your website is online and responding correctly. It monitors response times, SSL certificate validity, and HTTP status codes.
User Agent
VitalSentinel Uptime Bot/1.1 (+https://www.vitalsentinel.com/bot)robots.txt token
VitalSentinel-Uptime-BotThis is the token you would write in a User-agent: line. Please read “How to stop VitalSentinel traffic” below before you rely on it – a Disallow rule does not currently stop this fetch.
Request Frequency
Configurable by the website owner (1 minute or 5 minutes)
Behavior
- Makes GET requests to configured URLs
- Follows redirects (up to 10 hops)
- Default timeout of 10 seconds per request
- Verifies SSL certificate validity and expiration (hourly)
- Records response time, DNS lookup time, connect time, and TLS handshake time
- Does not execute JavaScript or load external resources
- Does not store or cache page content
VitalSentinel Bot
Scheduled robots.txt and sitemap checks
One name, shared by everything else that leaves our network for a third party. Two of the three reach your site: the robots.txt monitor, which reads /robots.txt and tells the owner when it changes, and the sitemap scanner, which reads the sitemaps a site declares. The third never touches your site at all - it reads Google's own status and announcements pages - so it is named here only so the string is fully accounted for. Because all three send the same user agent, the log line will not tell you which one called; the list below is what that name can mean.
User Agent
VitalSentinel Bot/1.1 (+https://www.vitalsentinel.com/bot)robots.txt token
VitalSentinel-BotThis is the token you would write in a User-agent: line. Please read “How to stop VitalSentinel traffic” below before you rely on it – a Disallow rule does not currently stop this fetch.
Request Frequency
On the schedule the site owner sets in the app, for the two that reach your site
Behavior
- The robots.txt monitor requests /robots.txt and nothing else, and compares it against the copy from the previous check
- The sitemap scanner reads the sitemaps a site declares, and the children of a sitemap index. It reads sitemaps, not the pages they list
- Neither follows links out of those documents, and neither crawls your pages
- The third service sending this name scrapes Google's status and announcement pages, so it never reaches a customer site and will never appear in your logs
- Fetches exactly what an anonymous visitor would receive - we do not log in, and we see nothing that is not publicly served
- Like every other service here, it does not consult a robots.txt Disallow rule before its own request, so a rule written for the token above will not stop it. Block the user agent at your CDN or WAF, or email us
Synthetic monitoring (Lighthouse)
Lighthouse performance audits and Core Web Vitals measurement
Synthetic tests run in two places. The default location runs on Google's PageSpeed Insights infrastructure, so those requests reach you from Google and carry Google's user agent, not ours. The additional locations available on higher plans run Lighthouse in headless Chrome on our own servers. Neither path sends a VitalSentinel-specific user agent, so there is no VitalSentinel string to match in your logs for these tests.
User Agent
None specific to VitalSentinel – see the description above.
Request Frequency
Configurable by the website owner (hourly, every 4/8/12/24 hours)
Behavior
- Runs Lighthouse in headless Chrome
- Default location runs on Google's PageSpeed Insights infrastructure
- Additional-location tests run from VitalSentinel's own servers
- Full Lighthouse audits (Performance, Accessibility, Best Practices, SEO)
- Tests both mobile and desktop viewports with realistic throttling
- Mobile: 2x CPU slowdown, 100ms RTT, 5 Mbps down / 2.5 Mbps up
- Desktop: No CPU slowdown, 20ms RTT, 25 Mbps down / 10 Mbps up
- Measures Core Web Vitals and lab metrics (LCP, CLS, FCP, TBT, TTI, Speed Index)
- Captures screenshots and filmstrip frames
- Default timeout of 120 seconds per test
VitalSentinel Free Tools Bot
The free tools at vitalsentinel.com/tools
This is the only VitalSentinel traffic that is not monitoring. It appears when a person pastes a URL into one of our free tools and presses a button – a robots.txt test, an indexability check, a redirect trace. Nothing about it is scheduled: no repeat visit, and no follow-up crawl the next day. Whoever ran the check does not have to own the site, in the same way that anyone can paste any URL into any online checker.
User Agent
Mozilla/5.0 (compatible; VitalSentinel Free Tools Bot/1.0; +https://www.vitalsentinel.com/bot)Request Frequency
On demand only – once per check a person runs by hand
What we keep from your page
One usage record per run is kept for 180 days and is then deleted automatically by a scheduled job. It holds the time of the run, the hostname that was checked – never the path, the query string or the full URL – the country the request came from, whether the check succeeded, how long it took, and a few small values from the result itself, such as a count, a grade, or the schema types found. It holds no IP address, no page content, and nothing that links one run to another. Beyond that record nothing about your site is stored: the result is computed and returned to the browser that asked for it, and a copy may sit in a short-lived edge cache – five minutes for most checks, fifteen minutes for sitemap and browser-rendered runs, an hour for Chrome UX Report data – so that repeat checks of the same domain do not cost that domain repeat requests.
Behavior
- Runs only in response to someone submitting a URL on a /tools page, and never on a schedule
- Fetches the URL that was submitted, plus a small fixed set of well-known paths that the tool in question needs – /robots.txt for the crawling tools, the sitemaps a sitemap index points at, a fixed list of /.well-known/ documents for the AI agent readiness check
- The list of extra paths is fixed per tool and the number of URLs fetched per run is capped, so a single check cannot fan out across your site
- Follows no links. It does not discover pages, and it does not walk your site
- Sends the user agent above whether the request is a plain HTTP fetch or a page loaded in a real browser, so there is one string to match in your logs rather than two
- Each request has a timeout, a cap on how many bytes of the response body are read, and a limit on redirect hops
- Obeys the same fair-use limits published at the bottom of every tool page: a per-minute, per-hour and per-day cap per IP address – counted per /64 network prefix for IPv6 callers – plus a per-site cap so the tools cannot be pointed at anyone as a load generator
- Fetches pages exactly as an anonymous visitor would – we do not log in, and we see nothing that is not publicly served
- When a run hits one of these limits, the limit is printed on the result rather than hidden, so nobody reads a capped answer as a complete one
How to stop VitalSentinel traffic
We would rather be straight with you than give you a rule that does nothing. Our monitoring services do not currently consult a robots.txt Disallow rule before making their own requests, so adding one will not stop them.
To stop the scheduled monitoring traffic today, block the user agents listed above at your CDN, WAF, or web server. One exception, because it would waste your time otherwise: an in-app page audit loads the page itself with an ordinary Chrome user agent, so no user-agent rule will catch that one request. The free page audit does not do this – it announces itself on every request. If you would rather we stopped the checks against your site from our side, email us.
The free tools are the one case where a user-agent rule is complete. Every request they make sends VitalSentinel Free Tools Bot, including the ones that load a page in a real browser, so blocking that user agent stops all of it. The free page audit is the same, under its own name VitalSentinel Inspector Bot: the plain fetch and the browser load both send it. Those requests only ever happen because someone typed your URL into a form, so there is nothing scheduled to turn off.
One thing worth knowing before you block: these checks run because one of our customers asked us to watch your site. If you block them, whoever that is stops receiving results.
Questions, or want something removed?
If you have questions about our bots, or about a page of yours that we fetched, email privacy@vitalsentinel.com. A person reads that address. If your message is a data protection request, the one-month response period under Article 12(3) GDPR starts when it arrives there.
If we audited a page of yours, you can ask us what we hold about that audit and ask us to delete it. Send the URL and the approximate date. We locate audits by hostname and delete them by hand, so there is no self-service route for this. Two things we want to be clear about: we cannot currently prevent a future audit of your site, because we have no blocklist, and deleting our stored copy does not undo our customer's decision to run the audit.
For anything else, contact us.
This is what watching a site looks like
These bots run for someone else's website right now. The same checks can run for yours – uptime, Core Web Vitals, search, and analytics in one place.