Home Pages of Several Websites are disappearing / reappearing in Google Index
-
Hi,
I periodically use the Google site command to confirm that our client's websites are fully indexed.
Over the past few months I have noticed a very strange phenomenon which is happening for a small subset of our client's websites... basically the home page keeps disappearing and reappearing in the Google index every few days. This is isolated to a few of our client's websites and I have also noticed that it is happening for some of our client's competitor's websites (over which we have absolutely no control).
In the past I have been led to believe that the absence of the home page in the index could imply a penalty of some sort. This does not seem to be the case since these sites continue to rank the same in various Google searches regardless of whether or not the home page is listed in the index.
Below are some examples of sites of our clients where the home page is currently not indexed - although they may be indexed by the time you read this and try it yourself. Note that most of our clients are in Canada.
My questions are:
1. has anyone else experienced/noticed this?
2. any thoughts on whether this could imply some sort of penalty? or could it just be a bug in Google?
3. does Google offer a way to report stuff like this?
Note that we have been building websites for over 10 years so we have long been aware of issues like www vs. non-www, canonicalization, and meta content="noindex" (been there done that in 2005). I could be wrong but I do not believe that the site would keep disappearing and reappearing if something like this was the issue. Please feel free to scrutinize the home pages to see if I have overlooked something obvious - I AM getting old.
site:dietrichlaw.ca - this site has continually ranked in the top 3 for [kitchener personal injury lawyers] for many years.
site:burntucker.com - since we took over this site last year it has moved up to page 1 for [ottawa personal injury lawyers]
site:bolandhowe.com - #1 for [aurora personal injury lawyers]
site:imranlaw.ca - continually ranked in the top 3 for [mississauga immigration lawyers].
site:canadaenergy.ca - ranks #3 for [ontario hydro plans]
Thanks in advance!
Jim Donovan, President
-
I just took the first domain you gave me I tested them on two tools you lack canonical's on all but the homepage for all three and all three failed the https://varvy.com test
- imranlaw.ca
- dietrichlaw.ca
- canadaenergy.ca
- burntucker.com past the Varvy test but has only one canonical https://cl.ly/hPdN https://cl.ly/hPoe
- bolandhowe.com is the probably the most affected it has way too many 200 code URLs canonical's pointing to the HTTPS however they should be using a 301 redirect See search engine land post below & these photos https://cl.ly/hPyM & https://cl.ly/hPUj
Preform a search and replace see: https://cl.ly/hPe6
- https://searchenginewatch.com/sew/how-to/2291162/seo-audit-findings-4-hidden-technical-problems-that-can-send-dangerous-signals-to-search-engines
- https://searchenginewatch.com/sew/how-to/2300520/technical-seo-for-nontechnical-people
I took the domainIn number three above and ran it through screaming frog I found no canonical's for all but one URL. Take a look at what most of the URLs appear like.
In addition found that you have a redirect chain photos below they should go straight to HTTPS://www.canadaenergy.ca
I would utilize HSTS as well this will help considerably. And adding canonical's
https://cl.ly/hPJd to https://cl.ly/hPr1 to https://cl.ly/hPyj
Domain number two
the same situation you have one canonical URL homepage nothing else has a canonical
domain number one imranlaw.ca same situation see below no canonical except for the homepage
| Address | http://www.imranlaw.ca/ |
| URL Encoded Address | http://www.imranlaw.ca/ |
| Status Code | 200 |
| Status | OK |
| Content | text/html; charset=ISO-8859-1 |
| Size | 13160 |
| Title 1 | Mississauga Immigration Lawyer & Canadian Citizenship Attorney |
| Meta Description 1 | Imran Khan - Canada Immigration lawyer and Canadian Citizenship attorney Mississauga Immigration Lawyer |
| H1-1 | Canadian Immigration & Naturalization Lawyer |
| H2-1 | Imran Khan Law Office offers Legal Services in Immigration Law and Real Estate Law Matters. |
| Meta Robots 1 | index,follow |
| Canonical Link Element 1 | http://www.imranlaw.ca/ |
| Word Count | 275 |
| Level | 1 |
| Inlinks | 28 |
| Outlinks |19
|
| Address | http://www.imranlaw.ca/contact |
| URL Encoded Address | http://www.imranlaw.ca/contact |
| Status Code | 200 |
| Status | OK |
| Content | text/html; charset=ISO-8859-1 |
| Size | 14503 |
| Title 1 | Mississauga Immigration Lawyer - Contact |
| Meta Description 1 | Imran Khan - Canada Immigration lawyer and Canadian Citizenship attorney Contact Imran Khan |
| H1-1 | Contact Imran Khan Law Office |
| Meta Robots 1 | index,follow |
| Word Count | 276 |
| Level | 2 |
| Inlinks | 28 |
| Outlinks | 17 |A few domains the ones above which are listed below as well fail to be able to be seen by a synthetic Googlebot. Are you running them all on the same server?
You have some domains and in .com and others that end in .ca if you are looking in Google.ca and have geo-targeted the .com domains to Canada you should see them there. However if you're looking in Google.com obviously you cannot geo-target .CA domains to the United States therefore they would not show up in .com unless very rarely.
Deep crawl and screaming frog are going to be a best friends on this one. Please let me know if I can be of more help
here are my findings using a basic tool
and put it into https://varvy.com
The results were
Findable links
Well formed static links not found.
Page has no findable links.
Guideline states: 'Ensure that all pages on the site can be reached by a link from another findable page.'
Learn about links and site hierarchy
HTTP headers
Page headers when accessed as Googlebot.
Headers:
pages could not be found
https://varvy.com/hierarchyandlinks.html
Same thing for imranlaw.ca
Findable links
Well formed static links not found.
Page has no findable links.
Guideline states: 'Ensure that all pages on the site can be reached by a link from another findable page.'
Learn about links and site hierarchy
For canadaenergy.ca
Findable links
Well formed static links not found.
Page has no findable links.
Guideline states: 'Ensure that all pages on the site can be reached by a link from another findable page.'
Learn about links and site hierarchy
Amount of links
Amount of links not excessive.
0 links found on page.
Guideline states: 'Limit the number of links on a page to a reasonable number (a few thousand at most).'
Considering the amount of links on a page
**I wouldUse a tool like deepcrawl.com or screamingfrog.co.uk/seospider **
two determined exactly what is wrong with all three Domains which failed a very basic test of being able to be detected by Googlebot.
Hope this helps,
Tom
-
Hi Jim,
If analytics confirms that traffic is still landing on the homepage, then I think this is just Google reporting different pages when you perform a site: - It certainly doesn't sound like a penalty of any sort.
It is worth noting that Google did confirm some time back that site: doesn't bring back every page every time and is best used as a guide. Does the sitemap in Search Console show a healthy number of indexed links?
If you want a discussion on this, then it would be worthwhile also posting over at the Websearch Help Forums at Google and see what others have to say about it.
I hope this helps a little.
-Andy
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
I want to move some pages of my website to a folder and nav menu in those pages should only show inner page links, will it hurt SEO?
Hi, My website has a few SaaS products, to make my website simple i want to move my website some pages to its specific folder structure , so eg website.com/product1/features
Technical SEO | | webbeemoz
website.com/product1/pricing
website.com/product1/information and same for product2 and so on, the website.com/product1/.. menu will only show the links of product1 and only one link to homepage (possibly in footer). Please share your opinion will it be a good idea, from UI perspective it will be simple , but i am not sure about SEO perspective, please help thanks0 -
Issues Indexing Translated Pages
I'm having trouble getting http://www.procloud.ch/ to index for their german pages. The english pages are being indexed but not the german. Any ideas? Chris
Technical SEO | | ninel_P0 -
Google Indexing of Site Map
We recently launched a new site - on June 4th we submitted our site map to google and almost instantly had all 25,000 URL's crawled (yay!). On June 18th, we made some updates to the title & description tags for the majority of pages on our site and added new content to our home page so we submitted a new sitemap. So far the results have been underwhelming and google has indexed a very low number of the updated pages. As a result, only a handful of the new titles and descriptions are showing up on the SERP pages. Any ideas as to why this might be? What are the tricks to having google re-index all of the URLs in a sitemap?
Technical SEO | | Emily_A0 -
Page Title Displaying differently on Google
Hi i am wondering why page title is display differently on google search. The combination of words that are being displayed on google are not on the page and the page title is simply "Camera Filters", however the page is showing as "our range of Camera Filters" (with the same capitalisation). http://awesomescreenshot.com/0cf4r09y27 I have optimised the age as best i can so it removes the OUR RANGE OF preface, however no luck. Any info would be appreciated. Cheers
Technical SEO | | andrewlos0 -
Pages Indexed Not Changing
I have several sites that I do SEO for that are having a common problem. I have submitted xml sitemaps to Google for each site, and as new pages are added to the site, they are added to the xml sitemap. To make sure new pages are being indexed, I check the number of pages that have been indexed vs. the number of pages submitted by the xml sitemap every week. For weeks now, the number of pages submitted has increased, but the number of pages actually indexed has not changed. I have done searches on Google for the new pages and they are always added to the index, but the number of indexed pages is still not changing. My initial thought was as new pages are added to the index, old ones are being dropped. But I can't find evidence of that, or understand why that would be the case. Any ideas on why this is happening? Or am I worrying about something that I shouldn't even be concerned with since new pages are being indexed?
Technical SEO | | ang1 -
Duplicate Home Page
Hi everyone! So, I;m using the crawl diagnostics in Moz and it's telling that I've got duplicate content for these two pages: http://www.bridgelanguages.com/
Technical SEO | | Bridge_Education_Group
http://www.bridgelanguages.com/index.php?p=3233&source=3 Would a redirect from the 2nd page to the 1st one be a solution? I'm not even sure where that 2nd link is on the site? Any suggestions or has anyone experienced the same? Thanks! Kelly0 -
Canonical tag for Home page: with or without / at the end???
Setting up canonical tags for an old site. I really need advice on that darn backslash / at the end of the homepage URL. We have incoming links to the homepage as http://www.mysite.com (without the backslash), and as http://www.mysite.com/ (with the backslash), and as http://www.mysite.com/index.html I know that there should be 301 redirects to just one version, but I need to know more about the canonical tags... Which should the canonical tag be??? (without the backslash) or (with the backslash) Thanks for your help! 🙂
Technical SEO | | GregB1230 -
Website disappeared from Google organic keyword searches.
We have an auto repair company as a client www.autorepairauroratilden.com who for the better part of a year their website had ruled the 1st page organic Google search results. Their website, Blogs, Facebook, and Twitter all came up on page one for their keyword searches. On May 13th, it all came to a screeching halt. The website is nowhere to be found for any of their keywords (example: brake repair Aurora.) There are a couple of blogs on page 2 but it’s nothing like it was prior to May 13th. On May 12th we published 5 branded websites for this client – Chrysler, Ford, Honda, Jeep, and Toyota, all on separate URL’s. All the page titles, keywords, and descriptions were specifically branded to the individual websites as were all the keywords. Since the beginning of June we’ve taken down the 5 branded websites and we’ve gone through our keywords on the auto repair website. The website was last crawled on June 11th. We still do not have any page 1 placement or for that matter any page placement. I checked 10 pages out. We have a 2nd auto repair client that has been running their website as well as their 5 branded websites a couple of months longer than this client and we’ve had no problems with any of their websites and keyword search results. How do we fix this?
Technical SEO | | markindenver0