Massive jump in pages indexed (and I do mean massive)
-
Hello mozzers,
I have been working in SEO for a number of years but never seen anything like a jump in pages indexed of this proportion (image is from the Index Status report in Google Webmaster Tools: http://i.imgur.com/79mW6Jl.png
Has anyone has ever seen anything like this?
Anyone have an idea about what happened?One thing that sprung to mind might be that the same pages are now getting indexed in several more google country sites (e.g. google.ca, google.co.uk, google.es, google.com.mx) but I don't know if the Index Status report in WMT works like that.
A few notes to explain the context:
- It's an eCommerce website with service pages and around 9 different pages listing products.
- The site is small - only around 100 pages across three languages
- 1.5 months ago we migrated from three language subdomains to a single sub-domain with language directories. Before and after the migration I used hreflang tags across the board. We saw about 50% uplift in traffic from unbranded organic terms after the migration (although on day one it was more like +300%), especially from more language diversity.
- I had an issue where the 'sort' links on the product tables were giving rise to thousands of pages of duplicate content, although I had used the URL parameter handling to communicate to Google that these were not significantly different and only to index the representative URL. About 2 weeks ago I blocked them using the robots.txt (Disallow: *?sort). I never felt these were doing us too much harm in reality although many of them are indexed and can be found with a site:xxx.com search.
- At the same time as adding *?sort to the robots.txt, I made an hreflang sitemap for each language, and linked to them from an index sitemap and added these to WMT. I added some country specific alternate URLs as well as language just to see if I started getting more traffic from those countries (e.g. xxx.com/es/ for Spanish, xxx.com/es/ for Spain, xxx.xom/es/ for Mexico etc). I dodn't seem to get any benefit from this.
- Webmaster tools profile is for a URL that is the root domain xxx.com. We have a lot of other subdomains, including a blog that is far bigger than our main site. But looking at the Search Queries report, all the pages listed are on the core website so I don't think it is the blog pages etc.
- I have seen a couple of good days in terms of unbranded organic search referrals - no spike or drop off but a couple of good days in keeping with recent improvements in these kinds of referrals.
- We have some software mirror sub domains that are duplicated across two website: xxx.mirror.xxx.com and xxx.mirror.xxx.ca. Many of these don't even have sections and Google seemed to be handling the duplication, always preferring to show the .com URL despite no cross-site canonicals in place.
Very interesting, I'm sure you will agree!
THANKS FOR READING!
-
Thanks for your considered response Adam. It is indeed quite possible that the jump is the non-English pages suddenly being indexed/reported as indexed in this WMT account. If there was to be a 'switch over' of the pages from one sub domain to the root domain, we would indeed have expected to see a jump like that.
It does still seem odd that (1) it came a long time after the migration and (2) the impressions and clicks (as reported in WMT) have not seen a similar jump, neither when the migration took place or in the last week. The 50% increase in clicks from unbranded organic I mentioned was a genuine increase, as our Analytics previously covered all three language sub-domains anyway.
On a side note, regarding the seperate subdomains, I was quite surprised to see how well the hreflang tags worked across sub domains before the migration. It was arguably better handled by Google before the migration to a single domain (more/better sitelinks for branded searches anyway). I think a lot of our uplift in clicks came from new pages and better on site optimization, and that the effect of consolidating the domains was not actually that big (in terms of clicks from unbranded organic). I think that the subdomain/directory debate is not quite as cut and dried as people think.
I must say, I love the hreflang tags - they are one of the most underrated tools in SEO in my opinion. Just don't forget that canonical tag or they don't work!
Thanks again for your reply!
-
Due to the information we have this response is obviously going to be some educated speculation. You said 1.5 months ago that you changed the structure in how you present your language options to the user and I think this has a great deal to do with the index pages your seeing.
If you check out Rand's SEO slideshare (http://www.slideshare.net/randfish/introduction-to-seo-5003433) from slides 39-47 you'll see his discussion on the importance of site structure in the eyes of Google. While translated content may be all the same to the user, the search engines take the structure to mean different matters of intent.
For example, sub-domain information is often taken to be duplicated translate purpose only content. It's also often categorized as a separate site.
When you went from sub-domains to language directories you went from three separate sites to one site with flow-down accessible information. In Google's eyes you just expanded your website with new fresh and valuable information. While some of the indexed pages may drop off I think this structural change is the main reason you've had such a pick up in indexing and hopefully it plays well for your on your international SEO campaign!
Cheers,
Adam
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Problem to get multilingual posts indexed on Google
Last year on June I decided to make my site multi-lingual. The domain is: https://www.dailyblogprofits.com/ The main language English and I added Portuguese and a few posts on Spanish. What happened since then? I started losing traffic from Google and posts on Portuguese are not being indexed. I use WPML plugin to make it multi-lingual and I had Yoast installed. This week I uninstalled Yoast and when I type on google "site:site:dailyblogprofits.com/pt-br" I started seeing Google indexing images, but still not the missing posts. I have around 145 posts on Portuguese, but on Search Console it show only 57 hreflang tags. Any idea what is the problem? I'm willing to pay for an SEO Expert to resolve this problem to me.
International SEO | | Cleber0090 -
Important pages are being 302 redirected, then 301 redirected to support language versions. Is this affecting negatively the linking juice distribution of our domain?
Hi mozzers, Prior to my arrival, in order to support and better serve the international locations and offering multiple language versions of the same content the company decided to restructure its URLs focused on locale urls. We went from
International SEO | | Ty1986
https://example.com/subfolder to https://example.com/us/en-us/new-subfolder (US)
https://example.com/ca/en-us/new-subfolder (CAN)
https://example.com/ca/fr-ca/new-subfolder (CAN)
https://example.com/de/en-us/new-subfolder (Ger)
https://example.com/de/de-de/new-subfolder (Ger) This had implications on redirecting old URLs to new ones. All important URLs such as https://example.com/subfolder were
302 redirected to https://example.com/us/en-us/subfolder and then 301 redirected to the final URL. According to the devs: If you change the translation to the page or locale, then a 302 needs to happen so you see the same version of the page in German or French, then a 301 redirect happens from the legacy URL to the new version. If the 302 redirect was skipped, then you would only be able to one version/language of that page.
For instance:
http://example.com/subfolder/state/city --> 301 redirect to {LEGACY URL]
https://example.com/subfolder/state/city --> 302 redirect to
https://example.com/en-us/subfolder/state/city --> 301 redirect to
https://example.com/us/en-us/new-subfolder/city-state [NEW URL] I am wondering if these 302s are hurting our link juice distribution or that is completely fine since they all end up as a 301 redirect? Thanks.1 -
Hreflang for selected pages?
My English site example.com has 1300 Pages I have launched a Russian transasation of the site example.ru This site has only 20 pages so far. Doubt: I need to set hreflang for all 1300 pages or only for 20 Pages of example.com that are converted to russian? Due to the limitations of the plugins avaialable I need to MANUALLY set hreflang for all 1300 pages of example.com Please guide
International SEO | | Janki990 -
Google does not index UK version of our site, and serves US version instead. Do I need to remove hreflanguage for US?
Webmaster tools indicates that only 25% of pages on our UK domain with GBP prices is indexed.
International SEO | | lcourse
We have another US domain with identical content but USD prices which is indexed fine. When I search in google for site:mydomain I see that most of my pages seem to appear, but then in the rich snippets google shows USD prices instead of the GBP prices which we publish on this page (USD price is not published on the page and I tested with an US proxy and US price is nowhere in the source code). Then I clicked on the result in google to see cached version of page and google shows me as cached version of the UK product page the US product page. I use the following hreflang code: rel="alternate" hreflang="en-US" href="https://www.domain.com/product" />
rel="alternate" hreflang="en-GB" href="https://www.domain.co.uk/product" /> canonical of UK page is correctly referring to UK page. Any ideas? Do I need to remove the hreflang for en-US to get the UK domain properly indexed in google?0 -
Country subfolders showing as sitelinks in Google, country targeting for home page no longer working
Hi There, Just wondering if you can help. Our site has 3 region versions (General .com, /ie/ for Ireland and /gb/ for UK), each submitted to Google Webmaster Tools as seperate sites with hreflang tags in the head section of all pages. Google was showing the correct results for a few weeks, but I resubmitted the home pages with slight text changes last week and something strange happened, though it may have been coincidental timing. When we search for the brand name in google.ie or google.co.uk, the .com now shows as the main site, where the sitelinks still show the correct country versions. However, the country subdirectories are now appearing as sitelinks, which is likely causing the problem. I have demoted these on GWT, but unsure as to whether that will work and it seems to take a while for sitelink demotion to work. Has anyone had anything similar happen? I thought perhaps it was a markup issue breaking the head section so that Google can no longer see the hreflangs pointing to each other as alternates. I checked the source code in w3 validator and it doesn't show any errors. Anyway, any help would be much appreciated - and thanks to anyone who gets back, it's a tricky type of issue to troubleshoot. Thanks, Ro
International SEO | | romh0 -
Include mobile and international versions of pages to sitemap or not?
My pages already have alternate and hreflang references to point to international and mobile versions of the content. If I add 5 language desktop versions and 5 language mobile versions as https://support.google.com/webmasters/answer/2620865?hl=en explains, my sitemap will get bulky. What are the pros and cons for referencing all page versions in sitemap and for include just general (English/Desktop) version in sitemap?
International SEO | | poiseo0 -
How fast is my front page?
Yesterday, I changed all of my front page structure from tables to divs. I think this has improved page load time, but I am in Australia, so it is hard to tell. Using Firefox with Firebug tells me the load time here is between 4 to 6 seconds. One of my editors is in Houston, and she says 2 seconds. I'm hoping you can help me, it will take less than a minute. Can you load the front page and tell me how long it takes - and where you are - Country/State Also, if you click to a story, how long does that take? http://newsblaze.com I am working on the story page template too, but it will take longer to get right, because it also is the same for 3 other areas, so I have to be more careful. It would also be nice to get a before and after snapshot from various places. The reason I care about shaving off a second or two is that I've been told google may now care about loading speed, and they are rejecting my new adsense account because of poor user experience on my site, and I have no idea what they mean by that, so I'm clutching at straws.
International SEO | | loopyal0 -
Geolocation and Indexing
Hi all, Our company owns site that have over 5 millions pages in Google index. We are locating in German, but our business aimed to US market. So, recently I checked index of our site using region targeting in US and there were only 150k of pages, but when I checked targeting in German there were almost 5 billion pages. Our server/IP locating in US, all the backlinks are from US sites. So, why there it is only small part of the site indexed in US? Regards, Dmitry
International SEO | | bubliki0