Measured on 197 websites
AI SEO: how AI assistants read your website
AI assistants do not read your page. They read one section of it. Someone types a prompt, the assistant fans that prompt out into several related searches and sends crawlers to run them, and those crawlers read your site section by section and take back the most relevant section. One section, on its own, with nothing either side of it.
Everything the answer needs has to be inside it. That single fact is what separates AI SEO from search engine optimisation, and it is why the page is no longer the unit you are writing for.
We measured 197 websites against that. Everything below is from those sites: 110 of them carry section-level counts, which is 15,592 sections.
13.2%
hand the crawler a blank page
Their pages contain no text at all until JavaScript runs. No major AI crawler runs JavaScript, so what it downloads is an empty shell. Google's crawler does run JavaScript, which is why one of these sites can rank normally, look perfect in a browser, and still give an assistant nothing to read. Count every site with any content hidden behind JavaScript and it is 21.3%, better than one in five.
Check this before anything else, because nothing further down this page can help a site that fails it. If the words are not in the HTML, no amount of rewriting them makes a difference.
Past that, five readings, in the order to act on them
Ordered by what each one costs you to fix, not by importance. The first is a five minute check that almost always passes, the second is markup, the third is a find and replace, and the last two are writing. Every percentage below is a failure rate measured across the 197 sites, not a score out of 100.
5.6%
of sites have a fault that blocks a crawler
01 Access
Can a crawler get in at all?
Almost everybody passes this, and the access reading is a perfect 100 on two thirds of the sites we measured. If you are reading a page like this one, your robots.txt is almost certainly fine. An llms.txt file is not the answer here either: 97% of the ones that exist are never requested by anything, and the sites that publish one still have half their sections under 40 words. It is a file, not a fix.
Fix. Check it once, confirm it is clean, and spend your time on the four below.
14.4%
of sections are a heading with no text under it
02 Chunking
Does the section stand on its own?
Mostly the web gets this right. Only 2.3% of sections open on a word like “this” or “however” that points at something the reader cannot see. The common fault is emptier than that. 97 of the 110 sites we could count have at least one heading with no text under it, and it is usually a grid of cards where every card title is marked up as a heading.
Fix. Any heading with under 5 words beneath it is either not a heading or is missing its answer.
76.6%
of sections never name the brand they belong to
03 Naming
Does the section name your website?
Your name is in the logo, the page title and the footer. None of those travel with the section. It arrives at the assistant saying “our platform helps teams move faster”, and nothing in it says whose platform that is.
Fix. Use your own name inside the section instead of “we” and “our platform”.
31 words
is the typical section on the typical site
04 Length
Is there enough there to take?
Common practice is to index in the region of 100 to 300 words, though that is practice rather than a rule and no best size has been published. Against it: 50.3% of all sections are under 40 words, and on 57.3% of sites most sections are. Sections that are too long are 2.0%, so the advice to break up walls of text is aimed at a problem almost nobody has.
Fix. Take your ten most important sections past 40 words. The section, not the page.
25.2%
of sections carry a number, a quotation or a link out
05 Evidence
Is there anything worth lifting?
The worst reading on this page by a distance. 4.8% of sections contain a quotation, 6.8% link to anyone else, and 18.2% of sites have no quotation anywhere on the site at all. An assistant answering a question needs something it can quote.
Fix. One checkable thing per section: a number with a unit, something someone actually said, or a link to a source that is not you.
Run an AI SEO audit on your site
It fetches your pages the way an AI crawler does, splits them into sections, and gives you your own reading on all five of the above. Free, a few seconds, and no account needed.
Questions people actually ask
What is AI SEO?
AI SEO is the work of making a website usable by AI assistants, not only by search engines. A search engine sends a person a link to your page. An assistant reads one section of your page and answers in its own words. That makes the section, rather than the page, the thing that competes, and it makes being readable by a crawler a prerequisite rather than a detail.
Is AI SEO different from SEO?
Less than most articles claim. Assistants lean heavily on ordinary search results, so search rank still matters a great deal. AI SEO adds two jobs on top of it. First, most AI crawlers never run JavaScript, so a page that renders in the browser can still arrive empty. Second, each section has to stand up alone, because it is handed over without the rest of the page around it.
Can AI assistants read my site?
Usually yes, and access is rarely the problem. Only 5.6% of the 197 sites Babel42 measured had any fault that stopped a crawler reading them at all. The more common failure is quieter: 13.2% of those sites returned no text until JavaScript ran, which means an AI crawler saw a blank page while the site ranked normally on Google.
Does ChatGPT read JavaScript?
No. Measured on production server logs, no major AI crawler runs JavaScript. The exceptions are Google's Gemini and Apple's AppleBot, which go through their own search systems. Anything on your page that only appears after scripts run is simply absent as far as most assistants are concerned.
Why is AI not citing my website?
The most likely answer is that nothing on the page is worth quoting. Across 15,592 measured sections, only 25.2% carried a number, a quotation or a link to another site, and 76.6% never named the brand whose site they were on. An assistant that cannot find a checkable sentence with an owner attached to it has no reason to name you.
Do I need an llms.txt file?
No, on the current evidence. Across 137,000 domains with traffic, 28% publish one and 97% of those files received no requests at all in a month. Google states its Search does not use them. Babel42's audit reports whether you have one and never counts its absence against you.
About the numbers. These are 197 websites whose owners chose to run a readability check, so they are not a random sample of the web and nothing here should be read as a share of all websites. 110 of them carry full section-level counts, which is 15,592 sections. The thresholds are ours: under 40 words is thin, over 350 is long, under 5 words beneath a heading is empty. They are reasonable, they are not standards, and another tool will draw the lines somewhere else.