International Sites - Sitemaps, Robots & Geolocating in WMT
-
Hi Guys,
I have a site that has now been launched in the US having originally just been UK. In order to accommodate this, the website has been set-up using directories for each country. Example:
As the site was originally set-up for UK, the sitemap, robots file & Webmaster Tools account were added to the main domain. Example:
The question is does this now need changing to make it specific for each country. Example:
The sitemap and robots.txt for the UK would move to:
and the US would have its own separate sitemap and robots.txt. Example :
Also in order to Geolocate this in WMT would this need to be done for each directory version instead of the main domain? Currently the WMT account for the UK site is verified at www.domain.com, would this need reverifying at domain.com/en-gb?
Any help would be appreciated! Thanks!
-
Thanks for the insights. Very helpful. What about the robots.txt, though? Should it stay under http://www.example.com, where crawlers can find the file?
-
Yes, that's how I did it.
-
Thanks for the response, much appreciated.
Would this mean that the sitemap featured at www.domain.com would need to reference each directory version (UK & US etc etc), leaving the en-gb & en-us to reference just their own?
-
Hi, here's what Google officially has to say:
Webmaster Tools data and reporting work best on a site level. For example, if your site http://www.example.com has separate sections for different countries, we recommend adding each of those subsites or subfolders as a separate site. For example, if you have a travel site with specific subfolders covering Ireland, France, and Spain, you could add the following sites to your Webmaster Tools account:
- http://www.example.com
- http://www.example.com/france
- http://www.example.com/ireland
- http://www.example.com/spain
Following this I have solved the exactly the same problem you had for a client. And it works perfectly.
- _The question is does this ["sitemap"] now need changing to make it specific for each country. _What I have done is add the sitemaps for each subdirectory as well as the main domain. It works (tracking and all that) and is easy to do.
- _Also in order to Geolocate this in WMT would this need to be done for each directory version instead of the main domain? _Yes, that is the way to go.
- _Currently the WMT account for the UK site is verified at www.domain.com, would this need reverifying at domain.com/en-gb? _Yes, it does. And so do every other subdirectory you add down the line. That allows precise international targeting per subdirectory via the hreflang and country targeting settings and setting the main GWT site for the root domain to be for global traffic.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Help with international targeting
Hi all! Okay, so we've got a site, let's say example.com - we sell training courses worldwide with a particular focus in just 8 countries.
International SEO | | JamieM1611
Historically, we've never targeted users in different countries effectively, we've just got the example.com that floats about ranking in different countries, but our content is dynamic (obviously a big SEO no-no - we pick up the IP of the user and show the content relevant for that country without the URL's changing)
This obviously presents an SEO flaw in that we can effectively target people in our key countries effectively. So, we're introducing the targeting as subfolders (/uk/, /ie/ etc) my questions are: 1. Would this be the correct implementation of hreflang AND canonical tags for the URL: https://www.example.com/es/ 2. The second thing I was wondering is the 'international targeting' in search console. We haven't (because of our current set up) set a target country for www.example.com (because of the lack of regional targeting and dynamic catch all) - would we be better leaving that untargeted and only specifying the regional targets for the new subfolder URLs (www.example.com/us/ /uk/ etc) or should we set the .com as the USA as default? We'd be a bit weary of doing this because most of our traffic comes from the UK and South Africa, so I'm assuming it would be best to leave this alone unless someone else has a different opinion? I know Googlebot almost always crawls .coms from US, which is why we were thinking of leaving the .com as the 'catch all' and specifying the US version. 3. Finally, we do have a lot pages which don't really change at all (like the about us page) would we give these any special directives to avoid duplicate content (as the content on these won't be changing at all?) or do we just keep the structure as shown above? I.E would the about us page (even though not changing) still be (with the canonical): URL: https://www.example.com/about-us/ (x-default) ? Thanks in advance!0 -
Correct Hreflang & Canonical Tags for Multi-Regional Website English Language Only having URL Parameters
Dear friends, We have a multi-regional website in English language only having the country selector on the top of each page and it adds countrycode parameters on each url. Website is built in Magento 1.8 and having 1 store with multiple store views. There is no default store set in Magento as I discussed with developer. Content is same for all the countries and only currency is changed. In navigation there are urls without url parameters but when we change store from any page it add parameters in the url for same page hence there are total 7 URLs. 6 URLs for each page (with country parameters) and 1 master url (without parameters) and making content duplicity. We have implemented hreflang tags on each page with url parameters but for canonical we have implemented master page url as per navigation without url parameters Example on this page. I think this is correct for master page but we should use URL parameters in canonical tags for each counry url too and there should be only 1 canonical tag on each country page url. Currently all the country urls are having master page canoncial tag as per the example. Please correct me if I am wrong and **in this case what has to be done for master page? **as google is indexing the pages without parameters too. We are also using GEOIP redirection for each store with country IP detection and for rest of the countries which are not listed on the website we are redirecting to USA store. Earlier it was 301 but we changed it to 302. Hreflang tags are showing errors in SEMRush due to redirection but in GWT it's OK for some pages it's showing no return tags only. Should I use **x-default tags for hreflang and country selector only on home page like this or should I remove the redirection? **However some of the website like this using redirection but header check tool doesn't show the redirection for this and for our website it shows 302 redirection. Sorry for the long post but looking for your support, please.
International SEO | | spjain810 -
International corporate SEO - menu and preferences
Hi all, for internationalization we have used subfolders structure (e.g. domain.com/us/) and since we have approx 150 countries to cover, then country selector is quite a link-base, especially if pages itself can have even several hundreds of internal links as well. Currently, all links to other countries are set as rel="nofollow" and I'm planning to change them to "follow" to distribute a juice. Do you please have an experience with such internal links voluminous websites and this follow/nofollow settings? Also we are having and issue with other countries outperforming local search. So for example in US, searching for kind of "Brand America" keyword, our UK branch outperforms US. Hreflangs are set in the sitemaps and each branch (country) is set in search console correctly as well. Since there are some issues with sitemaps (e.g. not actual links, or links returning 404), I assume that google is not using these sitemaps therefore hreflang information is lost as well. Am I correct ? thank you in advance, Tom
International SEO | | execom990 -
Google Analytics Search Console for International Countries
Hi Moz Community, Our e-commerce site is trying to gauge the opportunity of certain queries for specific countries. I'm trying to use the search console data presented in GA to do this. I'm looking at the top queries filtered by each country and also the top landing pages for each country as well. The non filtered data for queries and landing pages is completely different than by country and some if it looks wrong. For instance, our most popular query by impressions shows 0 query impressions in the US once filtered by country. Our site is based in the US so this doesn't make any sense, the same is true for landing pages. Is the queries and landing page data in GA under search console a combination of all countries? Since our target is set to the USA in search console is this data technically US based? How is this data so off? Thanks for answering!
International SEO | | znotes0 -
Is it compulsory to use hreflang attribute for Multilingual site? What if I do not use such tag?
Hello Everybody, My main site - abcd.co.uk and other sites are like this se.abcd.co.uk, fr.abcd.co.uk, es.abcd.co.uk etc Now if I donot use hreflang for Multilingual site then google will consider it as subdomain or duplicate site? But content of the sites are in different language. Thanks!
International SEO | | wright3350 -
Multinational Sites - The main SEO issues
I currently work for the UK arm of a Company with headquarters in Germany - The have outlets in half-a-dozen European countries, and up until now each country has had it's own website. The group has decided that from next year they will close all the individual country sites and then run new sites each from a central .location, I guess with a shared database of products. I see the sense in having central stock control etc, but I'm worried about the SEO impact. I have searched Q&A and the blog but could not find much to help me. What I would like to do is to provide some advice and pointers at to what they should be aiming for, both in terms website structure and on-going SEO for each country. Any advice welcome, thanks in advance.
International SEO | | cottamg0 -
Best Practice for International Website with Two Versions
I have a client in the medical industry, and the company's product has been approved in various countries in Europe yet is awaiting approval in the US. That means we can share certain information in some countries that we cannot share in the US. Therefore, we plan to use an initial landing page that will ask what country the user is in (using a drop-down list to choose from if not located in the US) and then push him or her to the appropriate version of the site. Here is my question: What is the best way to ensure search engines can crawl the site beyond this landing page? Thanks for your time.
International SEO | | mollykathariner_ms0 -
How should I make my site better?
I am glad to join seomoz,I am from China,the seomoz is a famous seo service provider company,some reason is one seoer guru named zac introduce seomoz to Chinese seor. So I think if seomoz provider seo tools or service to chinese seoer is a good idea.The market is very big.But chinese biggest SE is www.baidu.com,not google.There is something diffrent from baidu and google. My site is www.cn-sen.com, It's good performance at google with the keyword "除湿机",but it's have some trouble at baidu.I think the content of website is the main reason.and internal link is not good. Could someone give me some advise of seo to make my site better performance? thanks very much.
International SEO | | tylrr1230