Why isn't my site on Google?

A common question I get asked is: why isn't my site on Google? There are usually some simple answers to this question and this tutorial runs through most of them.

Finding out why a site isnt in the Google index can be done by a process of elimination.

Here are the steps that you can take to find out why a site isnt on Google:

Check to see if a site is in the Google index

The first thing that youll need to check is whether your site has been indexed by Google. The easy way to do this is by using the site: command in Google itself.

Go to Google and type in the URL for your site:

site:www.mysite.com

(If you want to check on a particular URL or page then you can type in site:mysite.com/a-page.html)

If Google returns no results then it means that youre not in Googles search index. If you have some results then it means that you are.

If youre not in the search results then youll need to proceed to the next steps to address this problem.

Check that youre not blocking the Google crawlers

Google sends out crawlers (or search engine spiders) around the Internet to find links to web pages. When it finds pages that it feels meet its quality guidelines then it adds them to its index. If youre blocking the crawlers then your site wont get indexed by Google.

There are two ways that your site can block search engine spiders:

1. By using a robots.txt file to block Google
2. By using a meta robots tag

A robots.txt file tells search engine crawlers which pages or files the crawler can or can't request from your site. It lives in the root directory of your website, for example, https://www.yoursite.com/robots.txt

If you have a robots.txt file and it contains something like this:

User-agent: Googlebot

Disallow: /

then it will block Google from indexing the site. If you want Google and all other search engines to access your site then simply remove the robots.txt file.

As mentioned above, the second way to block Google spiders from accessing your site is by using the meta robots tag. If youre using Wordpress then the creating/deletion of this can be managed from the admin interface.

The meta robot tag is located in the of the html document and takes the form:

<meta name='robots' content='follow,index'>

If your meta robot tag contains the words noindex then it wont be indexed. To get the site into Googles index, either remove the tag or replace the noindex directive with the word index. Youll then need to wait for Google to crawl your site and all the pages within.

Still no luck? Try spoofing the user agent

If the pages dont appear to be blocked and the page is still not indexed, try visiting the page whilst spoofing a Google user agent.

If you want to see what your site looks like from a search engine spiders perspective then there are several browser plugins that allow you to do this. One of these is the Firefox User Agent Switcher but there are others depending on the browser that you use.

Does Google know about your site?

If your website is not indexed and Googlebot is not blocked, the problem may be that Google does not know about your site.

If you have a Google Search Console account, you can remedy this by submitting your content to Google. A Google Search Console account is free to anyone with a Google account.

Is your content unique?

To find out if the content on your pages is unique, find some random text on your site and copy it. Now paste this into a Google search in quotes. If you find another site comes up for your search string it may mean that your content is duplicate content which Google doesnt like.

This will mean that the content for this page is more authoritative than yours and it may cause your site to not be indexed.

Do you have a sitemap?

If your website doesnt have a sitemap then you should think about adding one. Search engines such as Google like sitemaps as they provide an information hierarchy for the website and therefore make it easier for them to index.

Adding a sitemap to your Google Search Console account will ensure that Google knows about every page on your site.

Do you have links to your site?

In simple terms, ranking on Google is like a popularity contest. If your site is linked to by other sites then Google will see you as being authoritative.

If your site doesnt have enough links then this may be why you are not listed. To address this, add links to your site from pages that are already listed. You can also add internal links too from other pages on your site - internal links are not quite as good as external links but they indicate to Google that they are important pages.

If you still have no pages indexed after looking at all the above options then you may need extra help. You may have been affected by Google penalties or may have been bombarded by negative links. If this is the case, it may be wise to employ the services of a SEO specialist who can help repair your situation. Ive been working on search engine optimisation for over 20 years now and would be more than happy to give you some friendly advice based on your situation. If you need help, drop me a line and Ill see if I can help.