Allowing correct crawlers for GeoIP Redirect
-
Hi All,
I am working on an international site and we have started running into issues with crawlers successfully crawling the site.
GeoIPEnable On
Redirect one country
RewriteEngine on
RewriteCond %{ENV:GEOIP_COUNTRY_CODE} ^US$
RewriteCond %{HTTP:X-Host} !.nexcesscdn.net$ [NC]
RewriteRule ^(.)$ https://us.website.com/ [R,L]The main reason for working on a hard GEOIP redirect would be that we are unable to show certain products in certain regions, the customer should not be given the option which is best practice.
Can anyone advise?
Thanking in advance.
-
Actually, geo-basedIP redirects are still a very bad idea from a user and bot perspective.
- While Google has said it is testing crawling from other areas, they still primarilycrawl from the US. If you do Geo-basedredirects, they will only ever see the US content.
- Users travel. People travel. Assuming a user should only see a certain set of content based on their physical location is assuming too much.
- Use case in the consumer field: While attending a friend's wedding in London, I could not get to the US version of a site where I wanted to buy furniture to be delivered in a few weeks.
- Use case in business: Users travel for business all the time. If they are visiting a headquarters in another country but researching a topic for use in their home country, they might be seeing the "wrong content."
Rather than assuming, use IP detection to ask the user to set their location. "We see you are in the UK, do you want to set that as your preferred location?" Once they choose their location, a cookie is set and that is all that user sees from then on out, until they change that setting in the footer or in their account.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Googlebot being redirected but not users?
Hi, We seem to have a slightly odd issue. We noticed that a number of our location category pages were slipping off 1 page, and onto page 2 in our niche. On inspection, we noticed that our Arizona page had started ranking in place of a number of other location pages - Cali, Idaho, NJ etc. Weirdly, the pages they had replaced were no longer indexed, and would remain so, despite being fetched, tweeted etc. One test was to see when the dropped out pages had been last crawled, or at least cached. When conducting the 'cache:domain.com/category/location' on these pages, we were getting 301 redirected to, you guessed it, the Arizona page. Very odd. However, the dropped out pages were serving 200 OK when run through header checker tools, screaming frog etc. On the face of it, it would seem Googlebot is getting redirected when it is hitting a number of our key location pages, but users are not. Has anyone experienced anything like this? The theming of the pages are quite different in terms of content, meta etc. Thanks.
Intermediate & Advanced SEO | | Sayers0 -
How long should I keep the 301 redirect file
We've setup an new site and many pages don't exist anymore (clean up done). But for many of them we have new pages with new url's. We've monitored the 404 and have now many URL's redirected with 301 (apache file). How long should we keep this in place? Checking all links manually to see of new url is in place of the old url (in google) is too much work. tx!
Intermediate & Advanced SEO | | KBC0 -
How to handle individual page redirects on Wix?
I switched from one domain to another because I wanted a domain that had our company name so it was more brand-y. However, the old domain had better DA/PA. Originally I set up a global 301 from the old to the new, but now I'm finding that I actually need to set up individual 301's from each URL of the old site, or at least from each page. However, I am using Wix so it looks like I can't always do URL-URL 301's, although I can redirect any URL to a page on the new website. The problem is that, in some cases, the content on the new site is different (or, for example, I can only link a particular blog post on the old site back to the new site's blog's main page). How closely do URLS/pages need to resemble each other for link juice to be transferred? Also, should I try to set up all these redirects manually or bite the bullet and go back to using the old domain? The problem is that I did a lot of beginner SEO junk for the new domain, like submitting to a few higher-quality directories, and getting our website on various industry resource sites, etc. I'd need to re-do this entirely if I go back to the old page. What do you think?
Intermediate & Advanced SEO | | BohmKalish1230 -
301 redirects for a redesign.
About to completely redo a client's site and I want to make sure I don't loose our link juice. The current site is a old template site from another provider. They host it and we do not have access at all to the site itself, so there will be no transferring of the site from server to server because they feel the site is their property. Basically the site is a monthly service not a product. So this will be a completely new website, including new URL structure. So my question is how do keep the link juice flowing to the new site? I know I need to use 301 redirects, but do I rebuild those old URLs on my site and redirect them to their new counterpart or what? The link profile is not that impressive, maybe 15 back links (all mainly going to the homepage). But they all are local and coming from pretty good domain authority. But its keeping us ahead of our competition. Back story: This is one of my local search clients, we now have them ranking #1 across the board in the local packs. After analyzing the traffic, they are losing 75% of all traffic because of the sites design. So a new site is a must. I build a lot of websites, but have never worried about the back link profile before now. Thanks for all your help!
Intermediate & Advanced SEO | | masonrj0 -
How do you 301 redirect URLs with a hashbang (#!) format? We just lost a ton of pagerank because we thought javascript redirect was the only way! But other sites have been able to do this – examples and details inside
Hi Moz, Here's more info on our problem, and thanks for reading! We’re trying to Create 301 redirects for 44 pages on site.com. We’re having trouble 301 redirecting these pages, possibly because they are AJAX and have hashbangs in the URLs. These are locations pages. The old locations URLs are in the following format: www.site.com/locations/#!new-york and the new URLs that we want to redirect to are in this format: www.site.com/locations/new-york We have not been able to create these redirects using Yoast WordPress SEO plugin v.1.5.3.2. The CMS is WordPress version 3.9.1 The reason we want to 301 redirect these pages is because we have created new pages to replace them, and we want to pass pagerank from the old pages to the new. A 301 redirect is the ideal way to pass pagerank. Examples of pages that are able to 301 redirect hashbang URLs include http://www.sherrilltree.com/Saddles#!Saddles and https://twitter.com/#!RobOusbey.
Intermediate & Advanced SEO | | DA20130 -
Redirection strategy for mobile site
Hello folks! I am just about to launch a mobile specific version of our website. We were not able to make the main site responsive so have decided to make a seperate copy on an m dot subdomain. I have kept the url structure identical between both sites and added a canonical url on the mobile pages pointing to the desktop site. I will detect and redirect all mobile devices and googlebot mobile crawler to the m dot site. The questions i have are as follows... Is that the best approach if you use a mobile specific site on a seperate subdomain? What type of redirects should i use to send mobile users (and googlebot mobile) to the mobile site? My mobile site does not have all the pages the desktop site has. What happens if i redirect a mobile user from a page on the desktop site to a page on the mobile site that does not exist? (will give 404 currently). I guess i could maintain a list of valid mobile urls but this would be a pain (and a bit of an overhead) Your help is most appreciated Regards
Intermediate & Advanced SEO | | RobertHill0 -
Temporary Redirects on Magento
I've recently taken over a client who uses the Magento platform and there was definitely a duplicate issue with his homepage. It redirected www to non www, however the canonical tag was setup wrong and pointing to the www version. When I looked at OSE for both versions the non www has only 7 linking domains and a page authority of 32. The www version has 24 linking domains and page authority of 39. As the domain is fairly new, I decided to redirect the non www to www and keep the canonical the same. (I changed the internal linking structure etc). When I run both URLs through this tool: http://www.ragepank.com/redirect-... it's returning a whole bunch of 302, rather than 301 redirects. What's the deal with that? Is that a Magento setting that I can fix or something a little harder? I'm not sure if it's proper etiquette to post the URL of a client, so if that would help and is OK, please let me know. Thanks
Intermediate & Advanced SEO | | bradkrussell1 -
Htaccess Redirect with %C2%A0 in URL
Below is my setup for redirects in .htaccess file in my root word press installation. The www to non-www works well, so no problems there Other page redirects work well, too (example: redirect 301 /some-page/ http://mysite.com/another-page/ (I didn't post those because I have a few too many : ) So here it goes... RewriteEngine On
Intermediate & Advanced SEO | | pepsimoz
RewriteCond %{HTTP_HOST} ^www.mysite.com$ [NC]
RewriteRule ^(.*)$ http://mysite.com/$1 [R=301,L] BEGIN WordPress <ifmodule mod_rewrite.c="">RewriteEngine On
RewriteBase /
RewriteRule ^index.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]</ifmodule> END WordPress redirect 301 /archives/10-college- majors/ http://mysite.com/archives/10-college-majors/ redirect 301 /archives/10-college-%20majors/ http://mysite.com/archives/10-college-majors/ redirect 301 /archives/10-college-%C2%A0majors/ http://mysite.com/archives/10-college-majors/ I'm having a problem with the last 301 redirect: redirect 301 /archives/10-college-%C2%A0majors/ http://mysite.com/archives/10-college-majors/ not working... As you can see I've tried using other varations of the "space" but no go. I also used a redirect in cPanel's Redirect screen; testing all the possible options + wildcard I've also tried this: http://serverfault.com/questions/201829/using-special-characters-in-apache-mod-rewrite-rule (perhaps unsuccessfully, because it caused a 500 server error and it's a different situation in my case) I also saw something here: http://www.webmasterworld.com/apache/3908682.htm but I don't know if it works and how I would implement that + do so without compromising ALL other redirects. Note: the URL displays with a space in the address bar of all major web browsers: http://mysite.com/10-college- majors/ and goes to a 404 page I have a goregous page / PR6 / high authority site linking to the URL on my site, but they copied the URL with a space somehow. I contacted the person responsible for the website and he claims it works fine (aka he didn't check it). Is there a clean way to redirect ONLY this problematic URL without compromising other redirects, etc? Any ideas would be great. I'll respond with progress. Thanks in advance. UPDATE the redirect works, and it did work. Even so, when looking at source of page linking to mine, the URL looks like this: ``` http://mysite.com/archives/10-college- majors/ Clicking the URL in Source View in FireFox takes me to ``` http://mysite.com/archives/10-college-%C2%A0majors/ none of my 301 redirects should direct there. I don't have any redirect plugins either.0