Crawler transparency
CAI-Audit-Bot
CAI-Audit-Bot is the crawler used by Auditope to perform user-initiated audits on websites. It only runs when a logged-in user explicitly requests an audit on a URL — never autonomously, never on a schedule, never for indexing.
How to opt out
The simplest way is to add the following to /robots.txt:
User-agent: CAI-Audit-Bot
Disallow: /We re-read robots.txt at the start of every audit and respect Disallow rules strictly. If you prefer a permanent block list (independent of robots.txt), email abuse@auditope.eu with your domain; we’ll add it to the deny list and confirm via DNS-TXT verification.
Why this bot exists
Auditope is a self-serve site auditing platform. When a user runs an audit, we need to actually fetch and analyze the target site. This bot is the fetching component — single-purpose, user-initiated, transparent. It does not scrape content for resale, training, or indexing.
What it does
- Fetches the homepage and a small set of internal pages (sitemap-driven)
- Reads response headers, robots.txt, security.txt, manifest, structured data
- Renders the page in a real browser to compute Core Web Vitals + accessibility
- Stores results internally for the user’s report — not redistributed
What it does NOT do
- It does not submit forms, login, or interact with authenticated areas
- It does not bypass robots.txt, captcha, or rate-limit headers
- It does not persist private content (only public, non-authenticated pages)
- It does not share the data with third parties — we are EU-only and self-hosted
Suspect abuse?
If you see unusual traffic claiming to be CAI-Audit-Bot, contact abuse@auditope.eu with log lines (timestamp, User-Agent, source IP). We respond within 24h.