We Checked 37 UK Business Websites for Google and AI Search. Here Is What We Found.

We checked the public pages of 37 UK product and professional firms to see how ready they are for Google and for AI search. The short answer: almost none are blocking AI tools, so robots.txt is rarely the problem. Many do miss simple basics, such as one clear heading per page, that cost nothing to fix. This article shows what we found, how we found it, and what it cannot tell you.

What we did

In November 2026 we looked at 40 UK-based companies and could read 37 of them. Three blocked our requests or failed to load. The mix:

• 8 manufacturers
• 7 packaging companies
• 7 building materials and construction product firms
• 6 industrial suppliers
• 6 distributors and wholesalers
• 6 professional services firms (accountants, solicitors and consultancies)

We found each company with web searches using different search terms, and chose small and mid-sized firms, not household names. We did not include any Voltik clients. For each one we read the home page and the robots.txt file, and checked a few other standard files. We made light requests, a few seconds apart, and identified ourselves.

We name no companies in this article. The aim is the pattern, not to point at anyone.

The results

Share of the 37 reachable sites with each item on the home page, November 2026.

What we checked	Result
Mobile viewport tag	36 of 37 (97%)
Meta description	32 of 37 (86%)
Structured data (JSON-LD)	28 of 37 (76%)
Social sharing tags	28 of 37 (76%)
Canonical tag	26 of 37 (70%)
A sitemap.xml file	25 of 37 (68%)
Exactly one H1 heading	22 of 37 (59%)
Page title 60 characters or fewer	22 of 37 (59%)
An llms.txt file	9 of 37 (24%)
Blocks an AI training bot (GPTBot, ClaudeBot, Google-Extended)	2 of 37 (5%)
Blocks an AI search bot (OAI-SearchBot, Claude-SearchBot, PerplexityBot)	0 of 37
Hides a site from all robots with Disallow: /	0 of 37

Other findings:

• Images. 109 of the 2,022 images on home pages had no alt text, about 5%. But 11 of 36 sites (31%) had at least one image without it.
• JavaScript-only pages. One site had almost no readable text in its raw HTML, a sign that it relies on JavaScript to show content. The other 36 had plenty.
• Explicit AI rules. Four sites explicitly allowed at least one AI bot in robots.txt. Both sites that blocked training bots also blocked Common Crawl’s CCBot.

What this means

Blocking AI is not the common problem. If you worry that your robots.txt is hiding you from ChatGPT, check it, but the odds are that it is not. Most firms said nothing about AI bots, which leaves them open.

The basic fixes are cheap and often skipped. About four in ten sites did not have exactly one H1 heading, and about four in ten had a page title long enough to be cut off in results. Each takes minutes to fix.

Structured data is worth a look. About a quarter had none. It will not guarantee a result, and Google says AI features need no special markup. It does help search engines understand who you are and what you offer.

llms.txt got more attention than it has earned. Nine sites had an llms.txt file, more than had decided anything about AI bots in robots.txt. Google says it does not use the file, and studies have not found that it helps. A site is better off with a clear title and one H1.

The bigger gap is not on this list. These checks show whether a site is readable. They say nothing about whether the pages are clear, whether they show specs and proof, or whether other credible sites mention the company. Those decide whether you get named. See our AI search playbook for that work.

What this study cannot tell you

• It is a small, hand-picked sample. 37 firms found through search terms is not a random sample of UK businesses. Do not treat the percentages as national figures.
• We read home pages only. Inner pages may differ.
• We read the raw HTML. A site that adds tags with JavaScript after loading may show as missing something it has.
• We checked the standard address for the sitemap. Some sites use another address, so the 68% is a floor.
• robots.txt shows intent, not behaviour. We read the rules. We did not test whether each bot obeyed them.
• We did not measure rankings or AI answers. A site can be perfectly readable and still not be named.
• We left some measures out. We tried to count enquiry form fields and product-page tables, but our automatic checks picked the wrong pages too often, so we do not report them.

Check your own site in 10 minutes

• Open your home page and check it has exactly one H1 heading
• Check your page title is 60 characters or fewer and says what you do
• Add a meta description to every important page
• Open yoursite.co.uk/robots.txt and check it does not block the search bots
• Check yoursite.co.uk/sitemap.xml exists, or find where yours lives
• Look for structured data with Google’s Rich Results Test
• Give each image on your key pages alt text that describes it

We run these checks, and the deeper ones, for clients. Book a free call and we will look at yours. We plan to repeat this study every six months, so the numbers become a trend.

Sources

No outside sources for the results: they come from our own checks of 37 public websites in November 2026. For the context in this article:

• Google Search Central, “AI features and your website”.
• Google John Mueller, public statement on llms.txt, 17 June 2025.
• SE Ranking, llms.txt study of nearly 300,000 domains, October 2026.