Why Your Beautiful Website Doesn't Rank (Search Engines and AI Only See Code)

Just because a website looks good does not mean it is a good website. Search engines and AI tools cannot see your photography, your colour palette or your clever scroll animation. They see code, and they judge you on it. When we audit sites that were expensive, award-worthy and signed off by a whole committee, we regularly find the same thing underneath: a handsome shop front with the lights off inside. This article shows you how to look at your own site the way a crawler does, what usually goes wrong, and a 15-minute audit you can run today.

Quick answer

If your website looks great but does not rank, the usual cause is hygiene, not quality of design. Check four things: can crawlers reach the pages (robots.txt, CDN and plugin settings), does the important text exist in the HTML, is each page available at one clean URL with a canonical tag, and is the code free of template leftovers like wrong alt text and empty links. Fix those before you spend a penny on traffic.

Same page, two readers. Only one of them decides whether you get found.

Why does a good-looking website not rank on Google?

A good-looking site fails to rank when the code does not tell Google and AI tools what each page is about, who it is for and which URL is the real one. Design is for people. Rankings come from what crawlers can read, fetch and trust.

Think about what a crawler receives. It asks your server for a page and gets HTML. It reads headings, paragraphs, links, image descriptions, page titles and structured data. If your hero headline is a flat image, if your services sit in a slider that loads after a script runs, if your “Contact” button points at nothing, none of that design reaches the reader that matters here.

Here is the two-minute test. Open your home page, right click, choose View Page Source, and search for a sentence that appears on the page. If you cannot find it, the crawler may not find it either (in some cases Google can run scripts, but that is slower and less reliable, and many AI crawlers do not run them at all).

Do it from the command line if you prefer:

curl -s https://yourdomain.co.uk/ | grep -i "<h1"
curl -s https://yourdomain.co.uk/ | grep -i "<title"

You should see a real headline and a real title that describe the page in plain words. If you see nothing, or the headline reads “Welcome”, you have found work.

Common mistake: judging the site by how it looks on a laptop in the meeting room. Judge it by what the code says when you strip the design away.

Is your website builder or a plugin blocking AI?

Sometimes. Pages can be hidden from AI crawlers by a robots.txt rule, a CDN setting, a security plugin or a page builder that renders everything with scripts, and nobody on your team chose any of it. We call this the silent block, because the site looks fine to you.

Five places to check. Any one of them can hide a good site.

OpenAI documents three separate agents: OAI-SearchBot, which surfaces sites in ChatGPT search; GPTBot, which relates to training; and ChatGPT-User, which fetches pages when a person asks a question. Each is controlled independently in robots.txt, and OpenAI says a change can take about 24 hours to be picked up. If you block OAI-SearchBot, your site will not appear in ChatGPT search answers, though it can still show as a plain link.

Google says there is nothing special to do to appear in its AI features. A page needs to be indexed and eligible to show with a snippet, and Google’s advice is the old advice: allow crawling in robots.txt and through your CDN or hosting setup, link to content internally, keep important content as text, and make structured data match the visible page.

What to check, in order:

1. Your robots.txt. Visit yourdomain.co.uk/robots.txt. A healthy file for most business sites looks like this:

User-agent: *
Allow: /

User-agent: OAI-SearchBot
Allow: /

Sitemap: https://yourdomain.co.uk/sitemap.xml

If you see Disallow: / under User-agent: *, the whole site is closed. If you see lines naming GPTBot or OAI-SearchBot with Disallow, someone decided something. Make sure it was a decision you would still make. Blocking training (GPTBot) while allowing search (OAI-SearchBot) is a legitimate choice, and it is a choice, not an accident. 2. Your CDN or host. Cloudflare announced in July 2025 that it would block AI crawlers by default for new sites, with controls for existing customers. Other hosts and security tools have added similar switches. Log in and check the AI bots setting. If you cannot find it, ask your developer to confirm in writing. 3. Your security and SEO plugins. On WordPress, some plugins write AI-bot rules into robots.txt or block unknown user agents at the firewall. Re-read robots.txt after every plugin update. 4. Your page builder. Heavy builders can produce pages where text lives inside nested scripts and widgets. Check View Source as above. If text is missing, rebuild that section as plain headings and paragraphs. 5. Your login walls and pop-ups. A cookie wall or full-screen pop-up that fires before any content loads can stop a crawler seeing the page at all.

A legacy page builder on a content system, a one-and-done template builder, or an SEO plugin that was never set up for AI crawlers are the culprits we find most often. None of these is a villain. They are just tools that cap how much control you have over the code, and the control is what you are paying for.

What do website audits usually find?

Audits keep finding the same dozen hygiene problems, and almost all of them are invisible on screen. They come from templates, rushed launches and old versions of the site that never went away.

Wrong alt text copied from a template. The classic: a site that sells something completely different, with image descriptions about bed linen because the theme demo sold bed linen. Google calls alt text the most important attribute for images, and uses it with page context to understand what a picture shows. Wrong alt text tells it the wrong story.

Empty links and placeholders. The header “Contact” button pointing at an empty anchor (href="#"). A WhatsApp button with a placeholder and no phone number. A “Book a call” link that goes nowhere. The visitor clicks, nothing happens, and they leave. A crawler sees the same dead link.

The same page at two URLs. /services and /services/ both load the same content, with no canonical tag saying which is real. Google’s duplicate URL guidance says you can signal the preferred version with redirects, a rel="canonical" tag or your sitemap, and that these signals should agree. Google also says that if you specify nothing, it will usually choose a sensible version itself. That is true, and we still fix it: at best you waste crawling on duplicates, at worst links and signals get split between two addresses and the version Google picks is not the one you want.

Two domains for one brand. An old domain still live after a rebrand, or a second site for a second market. AI tools can read this as a split source of truth, with the old name treated as a separate company. Pick one, redirect the other, and claim the profiles so they all point to the same entity. We cover how to do that without losing rankings in our migration playbook.

Missing basics. No structured data, or thin FAQ markup. No clear statement of where the business is based, or company names that differ across directories and registries. Credentials and the founder story buried in an image instead of text. (Related: if your expert articles carry no named author, that is a trust problem we cover in E-E-A-T for professional services.)

Slow, heavy pages. Google’s “good” Core Web Vitals targets, measured at the 75th percentile of real visits, are Largest Contentful Paint at 2.5 seconds or less, Interaction to Next Paint at 200 milliseconds or less, and Cumulative Layout Shift at 0.1 or less. Beautiful sites often fail the first one because of one enormous hero image or video.

The pattern behind all of these: nobody owned the code after launch. The designer finished. The developer moved on. The marketing lead did not know where to look. Ranking, as we put it internally, is based on site hygiene.

How do you audit your own website in 15 minutes?

You can audit your own site’s hygiene in 15 minutes with a browser, Google Search Console and three short snippets. Work through the ten checks below, score yourself, and fix the lowest first.

Score each check 0 or 1. Under 7 means fix the foundations before spending on traffic.

Minutes 0 to 3: can it be reached?

1. Open /robots.txt. No Disallow: / under User-agent: *. No unintended AI-bot blocks. A sitemap line is present.
2. Open /sitemap.xml. It loads, lists your real pages with the right domain, and does not list redirected or noindexed pages.
3. In Search Console, open Settings, then Crawl stats. Crawl requests are healthy, and server errors are close to zero.

Minutes 3 to 8: does each page say the right thing?

4. View Source on your home page and your top service page. The <title> is unique and describes the page. There is one <h1>. Your main text is in the HTML.
5. Run this in the browser console (right click, Inspect, Console tab) on any page. It lists links that go nowhere:

[...document.querySelectorAll('a')].filter(a => ['','#'].includes(a.getAttribute('href')||'')).map(a => a.outerHTML.slice(0,120))

An empty list is the goal. Anything it prints is a dead link to fix. Also check the phone and WhatsApp links by hand: they should read tel: and https://wa.me/ followed by a real number. 6. Run this to find images with no description, then read the alt text on your top five images yourself:

[...document.images].filter(i => !i.alt).map(i => i.src)

Read the alt text aloud. Does it describe the picture and fit your business? If it mentions bed linen and you sell steel enclosures, you have found the template leftover.

Minutes 8 to 12: is there one clean version?

7. Type your home page four ways: http:// and https://, with and without www. All should land on one address in one step.
8. Add and remove a trailing slash on a service page URL. One version should redirect to the other, or both should carry a canonical tag pointing to the same address. Check with:

document.querySelector('link[rel=canonical]')?.href

It should return the full address of the page you are on. If it returns undefined, there is no canonical tag. 9. Search Google for site:yourdomain.co.uk. Skim the results for old pages, staging copies, odd parameter URLs and duplicates. Search your brand name and check no old domain ranks.

Minutes 12 to 15: can people act?

10. On your phone, on mobile data, tap every button in the header, the first screen and the footer. Submit your own enquiry form and confirm you get a response email. (See website lead capture for what to fix if you do not.)

Scoring. One point per passed check. 9 or 10: your hygiene is strong, so look at content and authority. 7 or 8: fix the failed items this week. Under 7: stop buying traffic until the site is fixed. We say this on audits because it is arithmetic: paid clicks and LinkedIn visitors landing on a leaky page cost the same as ones landing on a good page, and produce fewer enquiries.

What should you fix first?

Fix in this order, because each step makes the next one count: first anything that blocks crawling (robots, CDN, plugins, noindex), then dead or empty links and forms that do not work, then duplicate URLs and canonicals, then wrong or missing alt text and titles, then speed, then structured data, and last of all new content.

Why that order? A blocked site cannot benefit from good content. A form that does not submit wastes every visitor you earn. Duplicates and alt text are cheap. New content is the most expensive item, so you want it landing on a clean foundation. We put it as “fix the site first, then start driving people to it.” That rule applies to your LinkedIn activity and your ads as much as your SEO.

A realistic week of work:

Day	Task	Who
Mon	Robots, CDN and plugin check. Remove accidental blocks.	Developer
Tue	Fix dead links and test every form and phone link	Developer, owner tests
Wed	Redirect and canonical clean-up (www, https, slash, old domains)	Developer
Thu	Rewrite alt text on top 20 images. Fix titles and H1s on top 10 pages.	Marketing
Fri	Re-run the 15-minute audit and note the new score	Owner

Questions people ask

Why is my website not ranking on Google?

The most common causes are crawl blocks (robots.txt or noindex), pages that Google has not indexed, thin or duplicate content, slow pages, and a site that has few links or trust signals. Check Search Console’s Pages report first: it tells you which pages are indexed and why others are not.

Why is my website not showing up on Google at all?

Search site:yourdomain.co.uk. If nothing appears, check robots.txt for Disallow: /, view source for a noindex tag (on WordPress, check Settings, Reading), and make sure your site is verified in Search Console with a sitemap submitted. New sites can also simply take a few weeks to be found.

Does website design affect SEO?

Design affects SEO through what it does to the code: page speed, whether text is crawlable, whether mobile layout works, and whether navigation helps crawlers find pages. How a page looks is irrelevant to a crawler. How it is built is not.

Can ChatGPT and other AI tools read my website?

Only if they are allowed to fetch it and the content is in the HTML. Check that robots.txt and your CDN do not block OAI-SearchBot or other crawlers you want, and that your key facts (who you are, what you do, where, for whom) appear as plain text. Allow about 24 hours after a robots.txt change for OpenAI to adjust.

Does a website builder like Wix, Squarespace or Elementor hurt SEO?

Not automatically. The question is how much control you have over titles, headings, redirects, canonicals, structured data and robots rules, and whether your content lands in the HTML. Test with View Source and the audit above. If you cannot fix what the audit finds, that is the limit of the tool.

Want us to do this for you?

We run this audit, and a deeper six-area AI visibility audit, as the first step with clients, and we fix the site before we drive anyone to it. Book a free call and we will look at your site the way a crawler does, and tell you plainly what we find. If you want to read more first, our website service page shows how we build for both people and machines.

Sources

• Google Search Central, “Google Image SEO best practices”, Search Central documentation, accessed November 2026
• Google Search Central, “How to specify a canonical URL with rel=canonical and other methods”, Search Central documentation, accessed November 2026
• Google Search Central, “AI features and your website”, Search Central documentation, accessed November 2026
• Google web.dev, “Web Vitals”, Google, accessed November 2026
• OpenAI, “Overview of OpenAI Crawlers”, OpenAI developer documentation, accessed November 2026
• Search Engine Land, “Cloudflare to block AI crawlers by default with new pay per crawl initiative”, Search Engine Land, July 2025