There’s little point in publishing content that your target audience is never going to see.
And yet, this happens all the time – leading to wasted resources and opportunities.
When you publish new pages, it usually takes a few days to a few weeks for that content to be indexed by Google and other search engines. That page gets listed on a massive database – so users can find the content when searching for related information.
But some pages never get automatically indexed by search engines – or the indexing is removed.
This content is invisible to search engines, so people browsing the net can’t find it either.
Many business owners don’t realise when their pages aren’t indexed.
This article shows you how to quickly check that each web page has been indexed by Google (the most popular search engine). And if any pages aren’t indexed, we explain possible causes, and how to fix it.
This is crucial – so Google search bots can find your pages. You then have a shot at competing for the top-ranking spots that attract the most traffic because of high visibility.
But you have to be in the race to win it (only possible if indexed, of course)!
How to check if a page is indexed on Google
Here are two techniques you can use.
1. Check the indexing of a single page in Google:
Open a new window in “private browser” (incognito) mode
Type in site:page url with no spaces
If the page has been indexed, it will show up here:
If the page hasn’t been indexed, it will look like this:
2. Check the indexing of a single page in Google Search Console:
- Open your Google Search Console account (free to use)
- There’s a search bar at the top of the page: [inspect any URL in “website”]
- Type the exact URL of the page you want to inspect in that search bar
- Press enter
If your page isn’t indexed, you will see “URL is not on Google”.
3. Check what pages (multiple) haven’t been indexed:
Open your Google Search Console account and select “coverage” in the menu on the left side.
Tick the “error” box.
Any pages that were unable to be indexed will be listed (there are none in this example).
If any errors are listed, click on the error to find out what specific pages are impacted.
Give us a call if you don’t already have a free Google Search Console account, and need help setting one up (we plan to publish a guide on how to do this in the future).
Note:
All search engines have their own listings, so a page that’s indexed on Google isn’t necessarily indexed on others like Yahoo and Bing. However, usually pages that are indexed on one search engine eventually show up on the others too. If it is taking a long time you may want to contact us to sort it out.
Common reasons why your page isn’t indexed (and how to fix it)
1. The page was only recently published:
It usually takes Google between four days and four weeks to index a page. This duration is influenced by a number of factors, such as:
- Site domain age
- Level of site authority
- Website size
- Quality of website content
As a general rule of thumb, new and large websites take longer to index.
Is it possible to speed up this process?
Yes.
If you’d like your page to get indexed faster, you can ask Google to make this a priority.
Here’s how to do that in Google Search Console:
Enter the URL in the search bar at the top of the page
If the page hasn’t been indexed, you will see a message “URL is not on Google”. Click the “request indexing” button to get the page listed by Google.
Note: this particular technique only speeds up indexing on Google, as Google Search Console doesn’t link to other search engines.
By the way, it’s tempting to request indexing for every page, straight after publishing. However, don’t get carried away by submitting too many requests at the same time – as Google may not appreciate this influx. It’s best to use this tool if your page doesn’t get indexed after a reasonable amount of time (a week or so).
2. You accidentally set the page to “no index” when publishing:
You may not realise that “no index” tags are embedded in your code, while uploading content. This code may have been added by someone who designed your site.
If your page isn’t getting indexed, open the content management system that you’re using and check the code.
Find and remove:
- “noindex” meta tags in the pages HTML code
- “noindex” header tags in the HTTP request
A quick way to see if “noindex” codes are embedded into your code:
- CTRL + U (on windows) to open the page source code document
- CTRL + F search for “noindex”
If you don’t remove this code from pages that you want to index, Google crawlers will eliminate that page from the search results, and users won’t be able to find it.
Note: this strategy should resolve indexing for all search engines (not just Google)
3. The googlebot crawler is blocked in your robots.txt
The robots.txt file instructs crawlers to avoid or follow certain pages and elements on a website.
Has your file blocked Google’s crawlers from particular pages?
This doesn’t stop search engines from indexing the page, but they won’t crawl it – which means no page description will show up in the search results. However, this rarely happens (it would only be an issue if an SEO technician had previously set it up this way by mistake).
If you want to be sure:
Find out by opening your Google Search Console account and use this tool to identify any errors on a nominated URL.
Type the URL in the toolbar and click “test”
This is what you want the page to look like for indexing reasons:
A “blocked” description means that search engine crawlers have been told not to request that page from your site (to prevent your site from being overloaded with requests).
A faster way to find multiple blocked pages on your website:
Type /robots.txt after your URL in Google’s search bar.
Here’s an example from The Washington Post (all blocked pages are identified with “disallow”).
How to fix this:
You will need to access the robots.txt file on the server to edit it. If you’re using WordPress, SEO plugins like Yoast have functionality to fix this.
Note: this strategy should resolve indexing for all search engines (not just Google).
4. Your website has been penalised by Google:
Google may deindex your page if you breach its Quality Guidelines, spam links or break any other rules – whether intentionally or not. If your site has been hacked, you could also incur a penalty (even though this wasn’t your fault).
To find out if your site has been penalised, open the Manual Actions report on your Google Search Console account.
This is what you want to see:
If your website has been penalised, the issue will be listed in the report.
The penalty should be removed after you’ve fixed the violation and explained the resolution to Google.
Here’s a comprehensive list of Google penalties and how to resolve them (if you want to attempt this yourself).
Note: this strategy can have a positive impact on indexing with other search engines as well, as most search engines share similar rules.
Do you need extra support to index your pages?
If you’ve already tried everything, but your pages still aren’t being indexed, you may need assistance from SEO experts.
Call us on 1300 310 116 or fill out this form if you have any questions related to this article.
You can also book a free consultation to see how we can help you connect with your target audience online.