Googlebot being redirected but not users?
-
Hi,
We seem to have a slightly odd issue.
We noticed that a number of our location category pages were slipping off 1 page, and onto page 2 in our niche. On inspection, we noticed that our Arizona page had started ranking in place of a number of other location pages - Cali, Idaho, NJ etc. Weirdly, the pages they had replaced were no longer indexed, and would remain so, despite being fetched, tweeted etc.
One test was to see when the dropped out pages had been last crawled, or at least cached. When conducting the 'cache:domain.com/category/location' on these pages, we were getting 301 redirected to, you guessed it, the Arizona page. Very odd. However, the dropped out pages were serving 200 OK when run through header checker tools, screaming frog etc.
On the face of it, it would seem Googlebot is getting redirected when it is hitting a number of our key location pages, but users are not. Has anyone experienced anything like this? The theming of the pages are quite different in terms of content, meta etc.
Thanks.
-
Can anyone offer any insight on this? The issue shows no sign of improving, and our organic is tanking.
If this really is an issue on Google's side, and it appears based on our technical set up, how can I force Google to take note and reindex the pages?
Here are some more examples, from across a number of categories..
Category A:
Page that has dropped out of index: https://goo.gl/KEQ8Yh
Cached version of that page: https://goo.gl/DPzFWMPage that has dropped out of index: https://goo.gl/5KiQ4s Cached version of that page: https://goo.gl/myRWNg Category B: Page that has dropped out of index: https://goo.gl/pr3YQs
Cached version of that page: https://goo.gl/8SEYi5 Page that has dropped out of index: https://goo.gl/LqzDrg
Cached version of that page: https://goo.gl/iwPs45 CategoryPage that has dropped out of index: https://goo.gl/YBZS7c Cached version of that page:https://goo.gl/n33QzG Page that has dropped out of index: https://goo.gl/Ht4gfO
Cached version of that page: https://goo.gl/u81vbA -
Weirdly, at the time this seemed to correct itself overnight without any fixes our end. However 6 months on and it has happened again... Only this time, all location pages have dropped out of index, but latest cache versions all redirect to the Kansas page.
Have checked log files for these relevant URLs, and all show 200 (and being crawled regularly). When you fetch them in SC the show as complete, and they are all still indexed in Bing.
Very confused!
-
Hi Nicola,
Have you checked your canonical tags to be sure that they are pointing to the correct URL? Sometimes with storefront pages people just copy/paste the same HTML, swapping out the contact information. But maybe they forgot to update the canonical tags?
Keep us updated!
-
Thanks for advice Chris. Have checked, and can't see any issues with htaccess or robots seem. A very strange one indeed!
-
Hi Anthony,
Desktop returns 200, whilst mobile returns a 302 to the mDOT site.
Thanks.
-
Hi Nicola,
Sounds like a very strange one indeed. I can't say I've ever heard of Google giving a redirect when other 3rd party crawlers give you a 200.
My only suggestion to begin the troubleshooting would be to have a look at your htaccess and maybe Robots to see if there are any directives in there addressing Google specifically.
Anthony's suggestion is also a great idea. At least you'll see what Google is picking up on those pages right now rather what they saw last time it was cached.
-
You mention these pages are returning 200 in SEO tools, but would you please share what are these URLs are returning when you Fetch as Googlebot in Google Search Console? Test it in both desktop and mobile versions.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Best practice to redirect http to https
I have an SSL certificate on our domain but at times some search results still list the HTTP version. Clicking on this then warns the user about security and they leave. To avoid this I am using this in the htaccess file to redirect all HTTP visits to the https version. RewriteEngine On
Intermediate & Advanced SEO | | Gavpeds
RewriteCond %{SERVER_PORT} 80
RewriteRule ^(.*)$ https://domain.com/$1 [R,L] Is this ok? I notice on Moz toolbar it give a 302 temporary redirect I am thinking this isn't good and needs to be a 301 maybe? What is the best practice in this situation?1 -
Is it worth redirecting?
Hello! Is there any wisdom or non-wisdom in taking old websites and blogs that may not be very active, but still get some traffic, and redirecting them to a brand new website? The new website would be in the same industry, but not the same niche as the older websites. Would there be any SEO boost to the new website by doing this? Or would it just hurt the credibility of the new website?
Intermediate & Advanced SEO | | dieselprogrammers0 -
How many redirects are too many?
Hello Everyone, I currently have a dynamic site and it is my understanding that switching to a static site would be beneficial. I already have some 301's in place from when my site had a .php extension to the new extension now with ./?... etc. Is it okay to re redirect them? How many redirects are too many? Thank you in advance for suggestions. Have a Fabulous Friday! Sandra
Intermediate & Advanced SEO | | rankmenow0 -
Fetch as Google - Redirected
Hi I have swaped from HTTP to HTTPS and put a redirect on for HTTP to redirect to HTTPS. I also put www.xyz.co.uk/index.html to redirect to www.xyz.co.uk When I fetch as Google it shows up redirect! Does this mean that I have too many 301 looping? Do I need the redirect on index.html to root domain if I have a rel conanical in place for index.html htaccess (Linix) - RewriteCond %{HTTP_HOST} ^xyz.co.uk
Intermediate & Advanced SEO | | Cocoonfxmedia
RewriteRule (.*) https://www.xyz.co.uk/$1 [R=301,L] RewriteRule ^$ index.html [R=301,L]0 -
Geoip redirection, 301 or 302?
Hello all Let me first try to explain what our company does and what it is trying to achieve. Our company has an online store, sells products for 3 different countries, and two languages for each country. Currently we have one site, which is open to all countries, what we are trying to achieve is make 3 different stores for these 3 different countries, so we can have a better control over the prices in each country. We are going to use Geoip to redirect the user to the local store in his country. The suggested new structure is to add sub-folders as following: www.example.com/ca-en
Intermediate & Advanced SEO | | ajarad
www.example.com/ca-fr
www.example.com/us-en
... If a visitor is located outside these 3 countries, then she'll be redirected to the root directory www.example.com/en We can't offer to expand our SEO team to optimize new pages for the local market, it's not the priority for now, the main objective now is to be able to control the prices for different market. so to eliminate the duplicate issue, we'll use canonical tags. Now knowing our objective from the new URL structure, I have two questions: 1- which redirect should we use? 301, 302?
If we choose 301, then which version of the site will get the link juice? (i.e, /ca-en or /us-en?)
if we choose 302, then will the link juice remain in the original links? is it healthy to use 302 for long term redirections? 2- Knowing that Google bots comes from US-IP, does that mean that the other versions of the site won't be crawled (i.e, www.example.com/ca-fr), this is especially important for us as we are using AdWords, and unindexed pages will effect our quality score badly. I'd like to know if you have other account structure in your mind that would be better than this proposed structure. Your help is highly highly appreciated.
Thanks in advance.0 -
Redirection - Seo trick?
Hi, After analyzing the site I found several Redirections of exact match domains. With different domain name extensions. Is Seo trick? Is the second website which i fond that is using this technique. Can anyone gives more details? Thanks
Intermediate & Advanced SEO | | nyanainc0 -
301 redirect
I have 2 websites, lets call them Website A and Website B. Website A is a commercial website, website B is a 7 years old blog. Website B has many natural, high quality BL, including some from Nytimes, etc. I want to integrate the blog (B) into the commercial website (A). The idea behind this thought is to compress the two websites, it is easier to have everything in one place. I will do this with 301 redirect via Webmaster tools, htaccess etc. The uRL structure will remain the same: eg: websiteB/post-title/ -> websiteA/post title What will happen with my quality BLs? Is there any chance to be penalized by Google? What will happen with the PR of the 2 sites? Thanks.
Intermediate & Advanced SEO | | jasmin281 -
302 redirect
Aloha, I do a small study of 302 redirects. I wonder if you have any examples of sites where the use of a 302 is made.
Intermediate & Advanced SEO | | android_lyon
For example, to ski resorts: where there is a summer version and a winter version. In this case, the field of 302 will return the version of the relevant season. ex: http://www.valmorel.com/ >> 302 >> http://www.valmorel.com/fr/hiver/accueil-hiver.html I wonder if the use of 302 is the right solution.
What do you think? D.0