Skip to content
    Moz logo Menu open Menu close
    • Products
      • Moz Pro
      • Moz Pro Home
      • Moz Local
      • Moz Local Home
      • STAT
      • Moz API
      • Moz API Home
      • Compare SEO Products
      • Moz Data
    • Free SEO Tools
      • Domain Analysis
      • Keyword Explorer
      • Link Explorer
      • Competitive Research
      • MozBar
      • More Free SEO Tools
    • Learn SEO
      • Beginner's Guide to SEO
      • SEO Learning Center
      • Moz Academy
      • SEO Q&A
      • Webinars, Whitepapers, & Guides
    • Blog
    • Why Moz
      • Digital Marketers
      • Agency Solutions
      • Enterprise Solutions
      • Small Business Solutions
      • The Moz Story
      • New Releases
    • Log in
    • Log out
    • Products
      • Moz Pro

        Your all-in-one suite of SEO essentials.

      • Moz Local

        Raise your local SEO visibility with complete local SEO management.

      • STAT

        SERP tracking and analytics for enterprise SEO experts.

      • Moz API

        Power your SEO with our index of over 44 trillion links.

      • Compare SEO Products

        See which Moz SEO solution best meets your business needs.

      • Moz Data

        Power your SEO strategy & AI models with custom data solutions.

      NEW Keyword Suggestions by Topic
      Moz Pro

      NEW Keyword Suggestions by Topic

      Learn more
    • Free SEO Tools
      • Domain Analysis

        Get top competitive SEO metrics like DA, top pages and more.

      • Keyword Explorer

        Find traffic-driving keywords with our 1.25 billion+ keyword index.

      • Link Explorer

        Explore over 40 trillion links for powerful backlink data.

      • Competitive Research

        Uncover valuable insights on your organic search competitors.

      • MozBar

        See top SEO metrics for free as you browse the web.

      • More Free SEO Tools

        Explore all the free SEO tools Moz has to offer.

      NEW Keyword Suggestions by Topic
      Moz Pro

      NEW Keyword Suggestions by Topic

      Learn more
    • Learn SEO
      • Beginner's Guide to SEO

        The #1 most popular introduction to SEO, trusted by millions.

      • SEO Learning Center

        Broaden your knowledge with SEO resources for all skill levels.

      • On-Demand Webinars

        Learn modern SEO best practices from industry experts.

      • How-To Guides

        Step-by-step guides to search success from the authority on SEO.

      • Moz Academy

        Upskill and get certified with on-demand courses & certifications.

      • MozCon

        Save on Early Bird tickets and join us in London or New York City

      Unlock flexible pricing & new endpoints
      Moz API

      Unlock flexible pricing & new endpoints

      Find your plan
    • Blog
    • Why Moz
      • Digital Marketers

        Simplify SEO tasks to save time and grow your traffic.

      • Small Business Solutions

        Uncover insights to make smarter marketing decisions in less time.

      • Agency Solutions

        Earn & keep valuable clients with unparalleled data & insights.

      • Enterprise Solutions

        Gain a competitive edge in the ever-changing world of search.

      • The Moz Story

        Moz was the first & remains the most trusted SEO company.

      • New Releases

        Get the scoop on the latest and greatest from Moz.

      Surface actionable competitive intel
      New Feature

      Surface actionable competitive intel

      Learn More
    • Log in
      • Moz Pro
      • Moz Local
      • Moz Local Dashboard
      • Moz API
      • Moz API Dashboard
      • Moz Academy
    • Avatar
      • Moz Home
      • Notifications
      • Account & Billing
      • Manage Users
      • Community Profile
      • My Q&A
      • My Videos
      • Log Out

    The Moz Q&A Forum

    • Forum
    • Questions
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. Home
    2. SEO Tactics
    3. Intermediate & Advanced SEO
    4. Why would our server return a 301 status code when Googlebot visits from one IP, but a 200 from a different IP?

    Moz Q&A is closed.

    After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.

    Why would our server return a 301 status code when Googlebot visits from one IP, but a 200 from a different IP?

    Intermediate & Advanced SEO
    5
    16
    3282
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with question management privileges can see it.
    • danatanseo
      danatanseo last edited by

      I have begun a daily process of analyzing a site's Web server log files and have noticed something that seems odd. There are several IP addresses from which Googlebot crawls that our server returns a 301 status code for every request, consistently, day after day. In nearly all cases, these are not URLs that should 301. When Googlebot visits from other IP addresses, the exact same pages are returned with a 200 status code.

      Is this normal? If so, why? If not, why not?

      I am concerned that our server returning an inaccurate status code is interfering with the site being effectively crawled as quickly and as often as it might be if this weren't happening.

      Thanks guys!

      1 Reply Last reply Reply Quote 0
      • danatanseo
        danatanseo @Christy-Correll last edited by

        Howdie,

        Yes, I believe we got this sorted out. Interestingly, it wasn't any of the suggestions made here causing the 301 status code responses. I posted a thread in Google Webmaster Tools Forum regarding the issue and received a response that I am 99.5% sure is the correct answer.

        Here is a link to that thread for future readers' reference: https://productforums.google.com/forum/#!mydiscussions/webmasters/zOCDAVudxNo

        I believe the underlying issue has to do with incorrect handling of a redirect for this domain:  ccisound.com

        I am currently pursuing getting it corrected with our IT Director. Once the remedy is in place, I should know right away if it solves the issue I am seeing in the server logs. I'll post back here once I am 100% certain that was the issue.

        Thanks all! This has been an interesting one for me!

        1 Reply Last reply Reply Quote 1
        • Christy-Correll
          Christy-Correll Staff last edited by

          Hi Dana, have you definitively sorted this out?

          danatanseo 1 Reply Last reply Reply Quote 0
          • danatanseo
            danatanseo @ForForce last edited by

            They are pretty detailed, I'll send you yesterday's in a zip file so you can take a look. I'm certain that have everything needed. Thanks Eric!

            1 Reply Last reply Reply Quote 0
            • ForForce
              ForForce @William.Lau last edited by

              Right, a DNS manager could do a redirect, but that would not be visible in the web server log.  It would only be visible in whatever is managing the DNS.

              1 Reply Last reply Reply Quote 0
              • William.Lau
                William.Lau @danatanseo last edited by

                Depends what kind of DNS manager you are using. A redirect via DNS can still be possible.

                In my experience DNS managing software can redirect users with 301 or 302 headers depending on what settings you have. If your DNS manager has a security protocol along with redirect rules, it could be causing the issue.

                Examples of DNS redirects:

                https://dnsimple.com/url-forwarding-301-redirect

                https://support.cloudflare.com/hc/en-us/articles/200172286-How-do-I-do-url-forwarding-with-CloudFlare-

                ForForce 1 Reply Last reply Reply Quote 0
                • ForForce
                  ForForce last edited by

                  The request headers will also show if any and what cookies the user may have set.  Which it looks like is how your server determines if it should provide the client the desktop or mobile version.

                  1 Reply Last reply Reply Quote 1
                  • ForForce
                    ForForce last edited by

                    How detailed are your log files?  Can you see the user-agent (browser name) Maybe you could ask your IT department to log request headers?  If that will make the log files too big, they can probably do it only for the 'problem' IPs, or only for cases that the webserver returns a 301.  I'll take a look if you like.  Email is in my profile.

                    Best,

                    -Eric

                    danatanseo 1 Reply Last reply Reply Quote 1
                    • danatanseo
                      danatanseo @ForForce last edited by

                      Thanks so much Eric. Yes, I was thinking about the mobile version of our site being related to what I'm seeing too. However, I am unaware that we 301 redirect anything from the main site to the mobile site. In fact, users can actually switch to the mobile site via desktop by clicking "Mobile Site" in the footer and then browse the mobile version of the site via desktop. All of the URLs are identical.

                      Just out of curiosity I browsed to the mobile version of our site, grabbed a URL and then plugged it into "Fetch as Googlebot" in GWT. For all options, including desktop and the three mobile options a status code of 200 was returned.

                      1 Reply Last reply Reply Quote 0
                      • ForForce
                        ForForce last edited by

                        The problem can't be related to DNS.  If the problem was related to DNS, the request would never make it to your server, and you would never see anything related to the request in your log files.

                        Because you can see it in your log file, it is definitely happening on your own webserver (not some external problem).

                        The requesting IP is probobly not the problem, but it could be if your server automatically adds to a banned list any IP that requests > X pages in Y time - your server might think this is a DOS (denial of service) attack.... But if your server was set up to do this, your IT guys would probobly know about it.  This isn't something that is normally enabled 'out of the box' someone would need to intentionally activate a behavior like that.

                        More likely, is that there is another common denominator besides the requester IP...  I would guess that it's the user agent string (the browser or device the user is using).

                        Taking a quick look at what I think is your site, you have a mobile version available.  Google of course would be interested in what your site looks like to a mobile browser, and would send a 'fake' user agent string pretending to be so (a cell phone or a tablet etc...)  If your server sees this request, and tries to automatically redirect the browser to the mobile version of the site, then you would have your 301 code (which in this case is exactly what you intended, so your all set!)

                        There are probably a few other cases that could cause a 301 for just some IPs, but this is the only one that comes to mind at the moment.

                        Good Luck!

                        danatanseo 1 Reply Last reply Reply Quote 1
                        • danatanseo
                          danatanseo @William.Lau last edited by

                          Here is the response from my IT Director regarding the possibility that this is being done by our DNS manager:

                          "I do not believe so. Our DNS does translation of human readable names to IP address. It has nothing to do with the status being returned to a browser, and even if it did it could not write to the log file."

                          Is this accurate? I understand that the DNS cannot write to the log file, but if the DNS can flag a request to receive a certain status code from the server, then this scenario would still be a possibility.

                          William.Lau 1 Reply Last reply Reply Quote 0
                          • danatanseo
                            danatanseo @StreamlineMetrics last edited by

                            According to our IT Director we have no spam filters, no mod_security module, absolutely nothing on our server to prevent it from being crawled by bot, human or spider from any IP address, including black-listed IPs.

                            To me, other than the obvious (no security is probably not a good idea at all), that means that the 301 status codes being returned because of a problem with server set up.

                            I do have server logs that I'd be willing to share privately with anyone who's willing to take a gander. Don't worry, I won't send you a month's worth. 1-2 days should be plenty.

                            In the meantime I am going to dive in and take a look further. It's entirely possible that IPs from Google are not the only ones receiving nothing but 301 status codes in response to requests.

                            1 Reply Last reply Reply Quote 0
                            • danatanseo
                              danatanseo @William.Lau last edited by

                              Thanks William. Good suggestion. I am on it! I'll post back here once I know more.

                              1 Reply Last reply Reply Quote 0
                              • William.Lau
                                William.Lau @danatanseo last edited by

                                I would not be surprised if this was done by your DNS. If you use a DNS manager, they could possibly redirect certain users or IPs based on patterns of visits.

                                I suggest finding out more about any server configurations from the admin and seeing who they use as a DNS provider or manager.

                                danatanseo 2 Replies Last reply Reply Quote 1
                                • danatanseo
                                  danatanseo @StreamlineMetrics last edited by

                                  Excellent thoughts!  Yes, they are consistently the same IP addresses every time. There are several producing the same phenomenon, so I looked at this one 66.249.79.174

                                  According to what I can find online this is definitely Google and the data center is located in Mountain View, California. We are a USA company, so it seems unlikely that it is a country issue. It could be that this IP (and the others like it) are inadvertently being blocked by a spam filter.

                                  It doesn't matter the day or time, every time Googlebot attempts to crawl from this IP address our server returns 301 status codes for every request, with no exceptions.

                                  I am thinking I need to request a list of IP addresses being blocked by the server's spam filter. I am not a server administrator...would this be something reasonable for me to ask the people who set it up?

                                  Is returning a 301 status code the best scenario for handling a bot attempting to disguise itself as googlebot? I would think setting the server up to respond with a 304 would be better? (Sorry, that's kind of a follow-up "side" question)

                                  Let me know your thoughts and I'm going to go see if I can find out more about the spam filter.

                                  William.Lau 1 Reply Last reply Reply Quote 0
                                  • StreamlineMetrics
                                    StreamlineMetrics last edited by

                                    Where are the 301s taking Googlebot on those IP addresses? And are they the same IP addresses every time? Have you narrowed those IP addresses down to any particular datacenter/country? It could be possible there is some configuration with your server that treats IP addresses differently depending on the country... it could also be that the IP addresses getting the 301s are known blacklisted spam IP addresses but are masking themselves as Googlebot so your server's blacklist software is keeping them out. It's really hard to say without looking into the data myself but I'm definitely interested in what you find out.

                                    danatanseo 2 Replies Last reply Reply Quote 1
                                    • 1 / 1
                                    • First post
                                      Last post

                                    Got a burning SEO question?

                                    Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.


                                    Start my free trial


                                    Browse Questions

                                    Explore more categories

                                    • Moz Tools

                                      Chat with the community about the Moz tools.

                                    • SEO Tactics

                                      Discuss the SEO process with fellow marketers

                                    • Community

                                      Discuss industry events, jobs, and news!

                                    • Digital Marketing

                                      Chat about tactics outside of SEO

                                    • Research & Trends

                                      Dive into research and trends in the search industry.

                                    • Support

                                      Connect on product support and feature requests.

                                    • See all categories

                                    Related Questions

                                    • Sayers

                                      Googlebot being redirected but not users?

                                      Hi, We seem to have a slightly odd issue. We noticed that a number of our location category pages were slipping off 1 page, and onto page 2 in our niche. On inspection, we noticed that our Arizona page had started ranking in place of a number of other location pages - Cali, Idaho, NJ etc. Weirdly, the pages they had replaced were no longer indexed, and would remain so, despite being fetched, tweeted etc. One test was to see when the dropped out pages had been last crawled, or at least cached. When conducting the 'cache:domain.com/category/location' on these pages, we were getting 301 redirected to, you guessed it, the Arizona page. Very odd. However, the dropped out pages were serving 200 OK when run through header checker tools, screaming frog etc. On the face of it, it would seem Googlebot is getting redirected when it is hitting a number of our key location pages, but users are not. Has anyone experienced anything like this? The theming of the pages are quite different in terms of content, meta etc. Thanks.

                                      Intermediate & Advanced SEO | | Sayers
                                      0
                                    • msphoto

                                      Too many backlinks from one domain?

                                      I've been in the process of creating a tourism-based website for the state of Kansas. I'm a photographer for the state, and have inked a nice little side income to my day job as a web designer by selling prints from Kansas (along with my travels elsewhere). I'm still in the process of developing it, but it's at least at a point that I need to really start thinking about SEO factor of the amount of backlinks I have from it going back to my main photography website. The Kansas site is at http://www.kansasisbeautiful.com and my photography website is http://www.mickeyshannon.com. This tourism website will serve a number of purposes: To promote the state and show people it's not just a flat, boring place. To help promote my photography. The entire site is powered by my photography. To sell a book I'm planning to publish later this year/early next year of Kansas images. To help increase sales of photography prints of my work. What I'm worried about is the amount of backlinks I have going from the Kansas site to my photography site. Not to mention every image is hosted on my photography domain (no need to upload to two domains when one can serve the same purpose). I'm currently linking back to my site on most pages via a little "Like the Photos? Buy a print" link in the top right corner. In addition, when users get to the website map, all photo listings click back to a page on my photography site that they can purchase prints. And the main navigation also has a link for "Photos" that takes them to my Kansas photo galleries on my photography website as well. The question I have: Is it really bad SEO-wise to have anywhere from 1 to 10+ backlinks on every page from one domain (kansasisbeautiful.com) linking back to mickeyshannon.com? Would I be better served moving all of the content from kansasisbeautiful into a subdirectory on my photography site (mickeyshannon.com/kansas/) and redirecting the entire domain there? I haven't actually launched this website yet, so I'm trying to make the right call before pushing it to the public. Any advice would be appreciated!

                                      Intermediate & Advanced SEO | | msphoto
                                      0
                                    • Colin.Accela

                                      Setting up 301 Redirects after acquisition?

                                      Hello! The company that I work for has recently acquired two other companies. I was wondering what the best strategy would be as it relates to redirects / authority. Please help! Thanks

                                      Intermediate & Advanced SEO | | Colin.Accela
                                      0
                                    • RG_SEO

                                      Best server-side sitemap generators

                                      I've been looking into sitemap generators recently and have got a good knowledge of what creating a sitemap for a small website of below 500 URLs involves. I have successfully generated a sitemap for a very small site, but I’m trying to work out the best way of crawling a large site with millions of URLs. I’ve decided that the best way to crawl such a large number of URLs is to use a server side sitemap, but this is an area that doesn’t seem to be covered in detail on SEO blogs / forums. Could anyone recommend a good server side sitemap generator? What do you think of the automated offerings from Google and Bing? I’ve found a list of server side sitemap generators from Google, but I can’t see any way to choose between them. I realise that a lot will depend on the type of technologies we use server side, but I'm afraid that I don't know them at this time.

                                      Intermediate & Advanced SEO | | RG_SEO
                                      0
                                    • BrianAlpert78

                                      Can I make 301 redirects on a Windows server (without access to IIS)?

                                      Hey everyone, I've been trying to figure out a way to set up some 301 redirects to handle the broken links left behind after a site restructuring, but I can only ever find information on 2 methods that I can't use (as far as I can tell). The first method is to do some stuff with an htaccess file, but that looks like it only works on Linux-based servers. The method described for Windows servers is generally to install this IIS rewrite/redirect module and run that, but I don't think our web hosting company allows users to log directly into the server, so I wouldn't be able to use the IIS thing. Is there any other way to get a 301 redirect set up? And is this uncommon for a web hosting company to do, or do you all just run your sites on Linux-based servers or your own Windows machines? Thanks!

                                      Intermediate & Advanced SEO | | BrianAlpert78
                                      0
                                    • Travis-W

                                      Multiple 301 Redirects for the Same Page

                                      Hi Mozzers, What happens if I have a trail of 301 redirects for the same page? For example,
                                      SiteA.com/10 --> SiteA.com/11 --> SiteA.com/13 --> SiteA.com/14 I know I lose a little bit of link juice by 301 redirecting.
                                      The question is, would the link juice look like this for the example above? 100% --> 90% --> 81% -->72.9%
                                      Or just 100% -----------------------------------------> 90% Does this link juice refer to juice from inbound links or links between internal pages on my site? Thanks!

                                      Intermediate & Advanced SEO | | Travis-W
                                      0
                                    • ChatterBlock

                                      Serving different content based on IP location

                                      I have city centric website.  For sake of simplicity, say I only have 2 cities -- City A and City B. Depending on a user's IP address, they will either get City A or City B.  Users can change their location through javascript on pages.  But there is no cross-linking between cities.  By this, I mean that unless you can read or execute javascript, there is no way for you to get from city A to City B. My concern is this:  googlebot comes to my site, and we serve them up City A.  How does City B get discovered if Googlebot doesn't read javascript? We have an xml sitemap plus plenty of backlinks to City B.  Is this sufficient? Should I provide a static link to City B (and vice versa) on the homepage for crawling purposes?

                                      Intermediate & Advanced SEO | | ChatterBlock
                                      0
                                    • brianmcc

                                      Reverse Proxy better than 301 redirect?

                                      Are reverse proxies that much better than 301 redirects? Should I invest the time in doing this? I found out about reverse proxies here: http://www.seomoz.org/blog/what-is-a-reverse-proxy-and-how-can-it-help-my-seo

                                      Intermediate & Advanced SEO | | brianmcc
                                      0

                                    Get started with Moz Pro!

                                    Unlock the power of advanced SEO tools and data-driven insights.

                                    Start my free trial
                                    Products
                                    • Moz Pro
                                    • Moz Local
                                    • Moz API
                                    • Moz Data
                                    • STAT
                                    • Product Updates
                                    Moz Solutions
                                    • SMB Solutions
                                    • Agency Solutions
                                    • Enterprise Solutions
                                    Free SEO Tools
                                    • Domain Authority Checker
                                    • Link Explorer
                                    • Keyword Explorer
                                    • Competitive Research
                                    • Brand Authority Checker
                                    • Local Citation Checker
                                    • MozBar Extension
                                    • MozCast
                                    Resources
                                    • Blog
                                    • SEO Learning Center
                                    • Help Hub
                                    • Beginner's Guide to SEO
                                    • How-to Guides
                                    • Moz Academy
                                    • API Docs
                                    About Moz
                                    • About
                                    • Team
                                    • Careers
                                    • Contact
                                    Why Moz
                                    • Case Studies
                                    • Testimonials
                                    Get Involved
                                    • Become an Affiliate
                                    • MozCon
                                    • Webinars
                                    • Practical Marketer Series
                                    • MozPod
                                    Connect with us

                                    Contact the Help team

                                    Join our newsletter
                                    Moz logo
                                    © 2021 - 2025 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
                                    • Accessibility
                                    • Terms of Use
                                    • Privacy

                                    Looks like your connection to Moz was lost, please wait while we try to reconnect.