Babel42Bot
You saw us in your logs
Babel42Bot fetched your pages because somebody asked us to audit that site. Our audit tells a site owner whether AI assistants can read their pages, so the first thing it has to do is try to read them, exactly the way one of those assistants would.
We do not crawl continuously, we do not build an index, and nothing we read is used to train a model. We measure the pages, produce a report for the person who asked, and keep the numbers rather than your content.
The details you came for
- User agent
- Babel42Bot/1.0 (+https://babel42.io/bot)
- robots.txt name
- babel42bot
- Requests at once
- Three per site, at most
- Purpose
- One-off audits requested by a person, not continuous indexing
- JavaScript
- Never executed. We read the HTML your server sends
- Content use
- Measured, scored and discarded. Never used to train anything
How to stop us
Add this to your robots.txt. We obey it, and a blocked site is reported to whoever asked for the audit as exactly that: a site that asked us not to look.
User-agent: babel42bot Disallow: /
To keep us out of one part of the site only, name that path instead of the slash. If you would rather we left entirely and stayed gone, write to support@babel42.io and we will block the domain at our end as well.
If this is your site
You can run the audit yourself, free, and see everything we saw. It reports which AI crawlers your rules turn away, what your pages look like without JavaScript, and whether each section of a page can be quoted on its own.
Run the free audit