Home Pages of Several Websites are disappearing / reappearing in Google Index
-
Hi,
I periodically use the Google site command to confirm that our client's websites are fully indexed.
Over the past few months I have noticed a very strange phenomenon which is happening for a small subset of our client's websites... basically the home page keeps disappearing and reappearing in the Google index every few days. This is isolated to a few of our client's websites and I have also noticed that it is happening for some of our client's competitor's websites (over which we have absolutely no control).
In the past I have been led to believe that the absence of the home page in the index could imply a penalty of some sort. This does not seem to be the case since these sites continue to rank the same in various Google searches regardless of whether or not the home page is listed in the index.
Below are some examples of sites of our clients where the home page is currently not indexed - although they may be indexed by the time you read this and try it yourself. Note that most of our clients are in Canada.
My questions are:
1. has anyone else experienced/noticed this?
2. any thoughts on whether this could imply some sort of penalty? or could it just be a bug in Google?
3. does Google offer a way to report stuff like this?
Note that we have been building websites for over 10 years so we have long been aware of issues like www vs. non-www, canonicalization, and meta content="noindex" (been there done that in 2005). I could be wrong but I do not believe that the site would keep disappearing and reappearing if something like this was the issue. Please feel free to scrutinize the home pages to see if I have overlooked something obvious - I AM getting old.
site:dietrichlaw.ca - this site has continually ranked in the top 3 for [kitchener personal injury lawyers] for many years.
site:burntucker.com - since we took over this site last year it has moved up to page 1 for [ottawa personal injury lawyers]
site:bolandhowe.com - #1 for [aurora personal injury lawyers]
site:imranlaw.ca - continually ranked in the top 3 for [mississauga immigration lawyers].
site:canadaenergy.ca - ranks #3 for [ontario hydro plans]
Thanks in advance!
Jim Donovan, President
-
I just took the first domain you gave me I tested them on two tools you lack canonical's on all but the homepage for all three and all three failed the https://varvy.com test
- imranlaw.ca
- dietrichlaw.ca
- canadaenergy.ca
- burntucker.com past the Varvy test but has only one canonical https://cl.ly/hPdN https://cl.ly/hPoe
- bolandhowe.com is the probably the most affected it has way too many 200 code URLs canonical's pointing to the HTTPS however they should be using a 301 redirect See search engine land post below & these photos https://cl.ly/hPyM & https://cl.ly/hPUj
Preform a search and replace see: https://cl.ly/hPe6
- https://searchenginewatch.com/sew/how-to/2291162/seo-audit-findings-4-hidden-technical-problems-that-can-send-dangerous-signals-to-search-engines
- https://searchenginewatch.com/sew/how-to/2300520/technical-seo-for-nontechnical-people
I took the domainIn number three above and ran it through screaming frog I found no canonical's for all but one URL. Take a look at what most of the URLs appear like.
In addition found that you have a redirect chain photos below they should go straight to HTTPS://www.canadaenergy.ca
I would utilize HSTS as well this will help considerably. And adding canonical's
https://cl.ly/hPJd to https://cl.ly/hPr1 to https://cl.ly/hPyj
Domain number two
the same situation you have one canonical URL homepage nothing else has a canonical
domain number one imranlaw.ca same situation see below no canonical except for the homepage
| Address | http://www.imranlaw.ca/ |
| URL Encoded Address | http://www.imranlaw.ca/ |
| Status Code | 200 |
| Status | OK |
| Content | text/html; charset=ISO-8859-1 |
| Size | 13160 |
| Title 1 | Mississauga Immigration Lawyer & Canadian Citizenship Attorney |
| Meta Description 1 | Imran Khan - Canada Immigration lawyer and Canadian Citizenship attorney Mississauga Immigration Lawyer |
| H1-1 | Canadian Immigration & Naturalization Lawyer |
| H2-1 | Imran Khan Law Office offers Legal Services in Immigration Law and Real Estate Law Matters. |
| Meta Robots 1 | index,follow |
| Canonical Link Element 1 | http://www.imranlaw.ca/ |
| Word Count | 275 |
| Level | 1 |
| Inlinks | 28 |
| Outlinks |19
|
| Address | http://www.imranlaw.ca/contact |
| URL Encoded Address | http://www.imranlaw.ca/contact |
| Status Code | 200 |
| Status | OK |
| Content | text/html; charset=ISO-8859-1 |
| Size | 14503 |
| Title 1 | Mississauga Immigration Lawyer - Contact |
| Meta Description 1 | Imran Khan - Canada Immigration lawyer and Canadian Citizenship attorney Contact Imran Khan |
| H1-1 | Contact Imran Khan Law Office |
| Meta Robots 1 | index,follow |
| Word Count | 276 |
| Level | 2 |
| Inlinks | 28 |
| Outlinks | 17 |A few domains the ones above which are listed below as well fail to be able to be seen by a synthetic Googlebot. Are you running them all on the same server?
You have some domains and in .com and others that end in .ca if you are looking in Google.ca and have geo-targeted the .com domains to Canada you should see them there. However if you're looking in Google.com obviously you cannot geo-target .CA domains to the United States therefore they would not show up in .com unless very rarely.
Deep crawl and screaming frog are going to be a best friends on this one. Please let me know if I can be of more help
here are my findings using a basic tool
and put it into https://varvy.com
The results were
Findable links
Well formed static links not found.
Page has no findable links.
Guideline states: 'Ensure that all pages on the site can be reached by a link from another findable page.'
Learn about links and site hierarchy
HTTP headers
Page headers when accessed as Googlebot.
Headers:
pages could not be found
https://varvy.com/hierarchyandlinks.html
Same thing for imranlaw.ca
Findable links
Well formed static links not found.
Page has no findable links.
Guideline states: 'Ensure that all pages on the site can be reached by a link from another findable page.'
Learn about links and site hierarchy
For canadaenergy.ca
Findable links
Well formed static links not found.
Page has no findable links.
Guideline states: 'Ensure that all pages on the site can be reached by a link from another findable page.'
Learn about links and site hierarchy
Amount of links
Amount of links not excessive.
0 links found on page.
Guideline states: 'Limit the number of links on a page to a reasonable number (a few thousand at most).'
Considering the amount of links on a page
**I wouldUse a tool like deepcrawl.com or screamingfrog.co.uk/seospider **
two determined exactly what is wrong with all three Domains which failed a very basic test of being able to be detected by Googlebot.
Hope this helps,
Tom
-
Hi Jim,
If analytics confirms that traffic is still landing on the homepage, then I think this is just Google reporting different pages when you perform a site: - It certainly doesn't sound like a penalty of any sort.
It is worth noting that Google did confirm some time back that site: doesn't bring back every page every time and is best used as a guide. Does the sitemap in Search Console show a healthy number of indexed links?
If you want a discussion on this, then it would be worthwhile also posting over at the Websearch Help Forums at Google and see what others have to say about it.
I hope this helps a little.
-Andy
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Page disappears from Google search results
Hi, I recently encountered a very strange problem.
Technical SEO | | JoelssonMedia
One of the pages I published in my website ranked very well for a couple of days on top 5, then after a couple of days, the page completely vanished, no matter how direct I search for it, does not appear on the results, I check GSC, everything seems to be normal, but when checking Google analytics, I find it strange that there is no data on the page since it disappeared and it also does not show up on the 'active pages' section no matter how many different computers i keep it open. I have checked to page 9, and used a couple of keyword tools and it appears nowhere! It didn't have any back links, but it was unique and high quality. I have checked on the page does still exist and it is still readable. Has this ´happened to anyone before? Any thoughts would be gratefully received.0 -
Why Google de-rank a website.
Hi, I was inspecting a website which is covering the topic of best wheelbarrow of 2021, it is a new website and and starts ranking on google. But, after few days it got de-rank automatically and Moz is also not showing any result to that. I was wandering why this just happened and what should I do if I made my website and will not face this kind of situation?
Technical SEO | | Moeen22330 -
Stop Google indexing entire website based on search location
OK - bear with me... We have a .co.uk website. However, we only want it indexing in the US Google and NOT the UK Google. Is there a way of configuring this in Search Console /Webmaster tools?
Technical SEO | | AbsoluteDesign0 -
My sites "pages indexed by Google" have gone up more than qten-fold.
Prior to doing a little work cleaning up broken links and keyword stuffing Google only indexed 23/333 pages. I realize it may not be because of the work but now we have around 300/333. My question is is this a big deal? cheers,
Technical SEO | | Billboard20120 -
Google no longer is promoting my website!
My traffic just fell off the cliff and I can not figure out what has happened! Help!
Technical SEO | | PatriciaBeth0 -
Homepage no longer indexed in Google
Have been working on a site and the hompage has recently vanished from Google. I submit the site to Google webmaster tools a couple of days ago and checked today and the homepage has vanished. There are no no follow tags, and no robots.txt stopping the page from being crawled. It's a bit of a worry, the site is http://www.beyondthedeal.com
Technical SEO | | tonysandwich
Any insights would be massively appreciated! Thanks.0 -
How to determine which pages are not indexed
Is there a way to determine which pages of a website are not being indexed by the search engines? I know Google Webmasters has a sitemap area where it tells you how many urls have been submitted and how many are indexed out of those submitted. However, it doesn't necessarily show which urls aren't being indexed.
Technical SEO | | priceseo1 -
Will Google Continue to Index the Page with NoIndex Tag Upon Google +1 Button Impression or Click?
The FAQs for Google +1 button suggests as follows: "+1 is a public action, so you should add the button only to public, crawlable pages on your site. Once you add the button, Google may crawl or recrawl the page, and store the page title and other content, in response to a +1 button impression or click." If my page has NoIndex tag, while at the same time inserted with Google +1 button on the page, will Google recognise the NoIndex Tag on the page (and will not index the page) despite the +1 button's impression or clicks send signals to Google spiders?
Technical SEO | | globalsources.com0