What countries does Google crawl from? Is it only US or do they crawl from Europe and Asia, etc.?
-
Where does Google crawl the web from? Is it in the US only, or do they do it from a European base too? The reason for asking is for GeoIP redirection. For example, if a website is using GeoIP redirection to redirect all US traffic to a .com site and all EU traffic to a .co.uk site, will Google ever see the .co.uk site?
-
Hi Keith,
In my experience Google mainly crawls from the US.
You're quite right, GeoIP redirection can cause major issues with indexation - as if you're redirecting everything from the US to the .com googlebot can't see the .co.uk site.
As such, I'm not a fan. Rather than implementing a hard redirect I prefer Amazon's approach. If you visit the amazon.com site from a UK IP you get a javascript overlay that invites you to visit the .co.uk version of the site instead - they let the user decide which site to view rather than actually redirecting them.
This is a nice solution as it ensures that the search bots can crawl both versions of the site, and rankings aren't endangered.
I hope this helps,
Hannah
-
Keith, I am having the same issue and I agree with you. The fact that Google has data centers in Europe does not necessarily mean the algos are indexing from there. I also want to set Europe and US GeoIP redirection. It would be great to get Mozers opinions on this. Hopefully this post gets freshly reviewed
-
Interesting question - I'd quite like to know what happens here too.
Matt Cutts recently posted a video on cloaking (http://youtu.be/QHtnfOgp65Q) saying that as long as you don't do anything 'special' for Googlebot you're OK, but presumably if you are redirecting IP's based on location and you don't want to prevent Googlebot form accessing your site then effectively you have to do something 'special' for Googlebot (e.g. you're doing one thing for everyone else and a different thing for Googlebot).
-
Hi,
The data center locations is interesting, but it isn't what I was looking for. I need to know whether Google crawls the web from any IP other than US IPs.
To clear up the second question, let me be more specific:
Let's say Google is crawling a .co.uk site from a US IP address. The site is using GEO IP redirection to redirect all US traffic to the .com site. Therefore, when Google attempts to crawl the .co.uk from the US IP address site it will be redirected to the .com site, never seeing the .co.uk site. Can anyone confirm that this is what happens?
-
I found an article from 2008 that shows Google data centre locations: http://bit.ly/mONhf9
Your other question is a bit confusing. Why Google wouldn't see the UK site?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Which Google does a .ie website get shown in?
Hi all, I am working on a .ie website and I was under the understanding that if you have a regional domain, like .ie you will limited to being shown in a search engine like google.ie When I go to International Targeting in Google SEarch Console it says the site is associated with: Ireland Am I limiting my ability to rank well in worldwide Google searches with this domain and if so, how can I counter this? Many thanks.
International SEO | | Bee1590 -
How is Google determining location for geotargeting
Hello, I have a question related to Geotargeting.
International SEO | | Lvet
Let's suppose I have a website: mysite.co.uk
As far as I understand Google will consider this site targeted for the UK and will appear on searches preferentially from the UK. What happen now if I have a person located in Spain (with a Spanish IP) and searching in Google.co.uk. Will mysite.co.uk still appear in searches? What are the factors that Google takes into account for Geotargeting? Thank you0 -
Country subfolders showing as sitelinks in Google, country targeting for home page no longer working
Hi There, Just wondering if you can help. Our site has 3 region versions (General .com, /ie/ for Ireland and /gb/ for UK), each submitted to Google Webmaster Tools as seperate sites with hreflang tags in the head section of all pages. Google was showing the correct results for a few weeks, but I resubmitted the home pages with slight text changes last week and something strange happened, though it may have been coincidental timing. When we search for the brand name in google.ie or google.co.uk, the .com now shows as the main site, where the sitelinks still show the correct country versions. However, the country subdirectories are now appearing as sitelinks, which is likely causing the problem. I have demoted these on GWT, but unsure as to whether that will work and it seems to take a while for sitelink demotion to work. Has anyone had anything similar happen? I thought perhaps it was a markup issue breaking the head section so that Google can no longer see the hreflangs pointing to each other as alternates. I checked the source code in w3 validator and it doesn't show any errors. Anyway, any help would be much appreciated - and thanks to anyone who gets back, it's a tricky type of issue to troubleshoot. Thanks, Ro
International SEO | | romh0 -
Multi-Country Hosting
Hi all
International SEO | | yousayjump
I have read a few post that are similar to mine on this forum but still wanted to ask my specific question. We have a .com domain hosted on a dedicated server in the UK for our main website which takes online bookings and payments. We have purchased the following TLD's of our domain name:
.it, .es, .ch, .fr, .es, .at We essentially want to create a satellite site on each domain where the idea is that users in those countries will find our satellite sites via organic search engine results when searching for the information we sell. Due to the nature of our infrastructure, each satellite site will connect to the mysql database on the UK site to drawn down its text content. This is where the main CMS is located. I understand the following are important from an SEO point of view for our satellite sites: Hosting in the country we wish to target
Geographical targeting via WMT
Local addresses and telephone numbers on the site
Correct lang=" " tag in the source code
Content presented in the correct language (goes without saying really) Can you advise on any other considerations? Many Thanks Kris0 -
What is the best way to rank well on both Google.co.uk & Google.com?
I am working on a website that is primarily a UK based software company but is now expanding into the US. The website is a .com site and is not geo-targeted to any specific location. Currently the site ranks well on Google.co.uk for a number of the focus keywords. We are now targeting Google.com as well to increase visibility in the USA. The site is ranking number 1 for one of the focus terms on Google.co.uk but no where to be seen on Google.com but on another term the site ranks 3rd in both Google.co.uk and .com. There are a number of other terms that rank on the first page in Google.co.uk and on the 3rd or 4th page in Google.com. The server is located in Germany and I do not want to geotarget the site to the US as I am concerned this would have a negative impact on the .co.uk ranking. The site currently has a mix of .com and .co.uk links pointing back to the site, in actual fact possibly more links actually come from US sites already. My original plan was to just focus on building links back to the target pages from US sites rather than creating a US folder on the site and geotargeting that section of the site in WMT and having to build page authority for a completely new page with no existing backlinks. But now that I have a number 1 ranking on .co.uk and the same term not ranking at all in .com as well as a postion 3 ranking for a term in both .co.uk and .com I am slightly confused as to the best options. Any help, advice, opinions would be greatly appreciated.
International SEO | | PaulSimms0 -
Mini Google Search box within organic sitelinks
I'm trying to find solid information on Google's criteria on showing a "mini Google search box" within sitelinks? Example: if you type-in "wikipedia" on Google.com, a mini search box appears within the site links. This is for organic listing not paid listing. The Google Support page on Sitelinks does not state any info on the mini google search box. http://support.google.com/webmasters/bin/answer.py?hl=en&answer=47334
International SEO | | Mobies0 -
Geo Targeting SEO Techniques for Google UK
I'm starting a new SEO project whereby I'll be targeting UK search engines only such as Google.co.uk, (I'm from the states) and I'm gathering all the information I can get on this topic Obviously, I got a CO.UK TLD, and hosting/IP is UK based, but can anyone shed light on other techniques that has worked for you, Besides of the above here is some advice I picked up so far; Regional directory listings,
International SEO | | Plorex
Inbound and outbound inks from/to UK based websites,
Geographic targeting in Google webmaster tools,
British slang... What else is there?
Much appreciated0