Most Websites Score 39.5/100 on AI Search Readiness, Study Finds

A new benchmark scanning 6,382 websites found the average AI search readiness score is just 39.5 out of 100, and 28.8% of sites are unintentionally blocking legitimate AI crawlers. Sites with the highest readiness scores earned 72% more brand mentions in AI answers than the weakest ones, according to LightSite AI's State of AI Search Readiness report.

What changed?

LightSite AI, a Generative Engine Optimisation platform, scanned 6,382 unique domains between August 2025 and August 2026 to measure how well AI assistants can crawl, parse and answer from them, according to the State of AI Search Readiness report. The scan checked whether an AI assistant can reach a page, read its primary content in the initial HTML, extract structured data, and correctly identify the entity behind the site.

The results are stark. The average readiness score across all 6,382 domains was 39.5 out of 100, with the median site landing in the C band at a score of 47, and only 13.5% of sites scoring B or above.

Broken down by grade: 2.0% of sites (126) scored an A, 11.5% (736) scored a B, 38.7% (2,472) scored a C, 34.1% (2,178) scored a D, and 13.6% (870) scored an F. That means roughly half the web's brand-facing sites sit in the bottom two grades for AI readiness.

The single biggest fix identified in the study is crawler access. The report found that 28.8% of sites are blocking a legitimate AI crawler at the WAF or edge network, calling it the single highest-leverage fix available to site owners. This echoes an earlier LightSite study covered by Barchart, which found that 47% of websites have issues in their robots.txt files, often lacking clear allow rules for trusted AI agents or unintentionally blocking them through Cloudflare and other security settings.

Why does this matter for your business?

The headline finding is that technical readiness isn't a nice-to-have, it's a measurable driver of visibility. Sites that moved from the lowest technical band to the highest saw a 55% lift in AI visibility score, from 25.9 to 40.2, and a 72% lift in average brand mentions in AI answers, from 2.2 to 3.8, per the LightSite report.

In plain terms: if two competing brands have similar content quality, the one that's easier for an AI crawler to reach, parse and understand will get cited and recommended more often. The report states that sites AI crawlers can reach, parse, and extract from get retrieved more often, cited more often, and recommended more often, with the lift measurable across every layer of the sites studied.

This matters because most marketing teams still think about AI visibility purely in terms of content and messaging. This research suggests that before you even get to content strategy, a large chunk of the web is failing at the infrastructure layer: crawler access, structured data and clear entity signals. If your firewall or CDN is quietly blocking GPTBot, ClaudeBot or PerplexityBot, no amount of well-written content will help, because the AI system never sees the page in the first place.

The findings line up with a broader pattern seen elsewhere this year. Semrush's 2026 AI Visibility Index, based on an analysis of 126 million US AI search prompts, found that among organisations that fully integrate SEO and AI visibility into a unified workflow, 81% reported increased traffic or leads from AI platforms, compared with only 36% among those managing the two areas separately. Readiness and integration, not content volume alone, appear to be the recurring theme in 2026's visibility research.

What should you do now?

Start with the basics that this study flags as highest-leverage. Check your robots.txt file and your WAF or CDN settings, such as Cloudflare or Akamai, to confirm you aren't blocking recognised AI crawlers like GPTBot, ClaudeBot, PerplexityBot and Google's AI Overviews crawler.

Next, audit whether your key pages actually contain their core content in the initial HTML, rather than loading it in via JavaScript after the page renders. Many AI crawlers don't execute JavaScript the way a browser does, so content that only appears after a script runs may be invisible to them.

Then check your structured data. Schema markup and clear entity signals, meaning unambiguous information about who your company is, what it does and how it differs from competitors, help AI systems describe your brand accurately rather than guessing or defaulting to a competitor's framing.

Finally, treat this as an ongoing process rather than a one-off fix. Given how much variance the report found between the top and bottom bands, it's worth checking regularly rather than once a year. If you want a quick read on where your own site stands before committing time to a full audit, running it through Sited's free AI visibility audit gives you a starting benchmark against exactly these kinds of readiness signals.

Frequently asked questions

What is "AI search readiness"?

It refers to how easily an AI assistant, such as ChatGPT, Gemini or Perplexity, can access, read and understand a website. It covers whether an AI assistant can reach the page, read primary content in the initial HTML, extract structured data, and identify the entity behind the site without guessing.

How many websites did the LightSite study cover, and how were they scored?

The study scanned 6,382 unique domains between August 2025 and August 2026, grading each on a five-band scale from A to F based on technical crawlability and structured data completeness.

What's the single most common technical problem found?

Blocking AI crawlers at the network level is the most widespread and highest-impact issue. 28.8% of sites are blocking a legitimate AI crawler at the WAF or edge network, making it the single highest-leverage fix identified in the report.

Does fixing technical readiness actually increase AI mentions, or is it just theory?

The data suggests a real, measurable link. Moving from the lowest to the highest readiness band was associated with a 55% lift in AI visibility score and a 72% lift in average brand mentions in AI answers.

Is this just about robots.txt, or is there more to it?

Crawler access is the starting point, but not the whole picture. The report also weighs whether content loads in the initial HTML, whether structured data like JSON-LD is present, and whether the site clearly identifies the entity behind it, so fixing robots.txt alone won't guarantee a top score.