Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
How to stop Google crawling after 301 redirect?
-
I have removed all pages from my old website and set 301 redirect to new website. But, I have verified old website with Google webmaster tools' HTML verification file which enable me to track all data and existence of pages in Google search for my old website. I was assumed that, Google will stop crawling and DE-indexed all pages after 301 redirect. Because, I have set 301 redirect before 3 months.
Now, I'm able to see Google bot activity on my website with help of Google webmaster tools. You can find out attachment to know more about it. How can it possible & How Google can crawl removed pages?
You can see following image to know more about it.
&
-
Google is most likely following links on other sites pointing to your old site and then 301'ing to the new site so you're seeing activity in WMT
looking here is still see two pages in the index:
you can go in and remove the site in WMT using the remove URL tool and see if that stops activity in that old WMT account. Crawling or not crawling, reporting or not reporting, there is not an issue here though - the 301's appear to be properly set up.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
302 > 302 > 301 Redirect Chain Issue & Advice
Hi everyone, I recently relaunched our website and everything went well. However, while checking site health, I found a new redirect chain issue (302 > 302 > 301 > 200) when the user requests the HTTP and non-www version of our URL. Here's what's happening: • 302 #1 -- http://domain.com/example/ 302 redirects to http://domain.com/PnVKV/example/ (the 5 characters in the appended "subfolder" are dynamic and change each time)
Intermediate & Advanced SEO | | Andrew_In_Search_of_Answers
• 302 #2 -- http://domain.com/PnVKV/example/ 302 redirects BACK to http://domain.com/example/
• 301 #1 -- http://domain.com/example/ 301 redirects to https://www.domain.com/example/ (as it should have done originally)
• 200 -- https://www.domain.com/example/ resolves properly We're hosted on AWS, and one of my cloud architects investigated and reported GoDaddy was causing the two 302s. That's backed up online by posts like https://stackoverflow.com/questions/46307518/random-5-alpha-character-path-appended-to-requests and https://www.godaddy.com/community/Managing-Domains/My-domain-name-not-resolving-correctly-6-random-characters-are/td-p/60782. I reached out to GoDaddy today, expecting them to say it wasn't a problem on their end, but they actually confirmed this was a known bug (as of September 2017) but there is no timeline for a fix. I asked the first rep I spoke with on the phone to send a summary, and here's what he provided in his own words: From the information gathered on my end and I was able to get from our advanced tech support team, the redirect issue is in a bug report and many examples have been logged with the help of customers, but no log will be made in this case due to the destination URL being met. Most issues being logged are site not resolving properly or resolving errors. I realize the redirect can cause SEO issues with the additional redirects occurring. Also no ETA has been logged for the issue being reported. I do feel for you since I now understand more the SEO issues it can cause. I myself will keep an eye out for the bug report and see if any progress is being made any info outside of this I will email you directly. Thanks. Issue being Experienced: Domains that are set to Go Daddy forwarding IPs may sometimes resolve to a url that has extra characters appended to the end of them. Example: domain1.com forwards to http://www.domain2.com/TLYEZ. However it should just forward to http://www.domain2.com. I think this answers what some Moz users may have been experiencing sporadically, especially this previous thread: https://moz.com/community/q/forwarded-vanity-domains-suddenly-resolving-to-404-with-appended-url-s-ending-in-random-5-characters. My question: Given everything stated above and what we know about the impact of redirect chains on SEO, how severe should I rate this? I told my Director that I would recommend we move away from GoDaddy (something I don't want to do, but feel we _**have **_to do), but she viewed it as just another technical SEO issue and one that didn't necessarily need to be prioritized over others related to the relaunch. How would you respond in my shoes? On a scale of 1 to 10 (10 being the biggest), how big of a technical SEO is this? Would you make it a priority? At the very least, I thought the Moz community would benefit from the GoDaddy confirmation of this issue and knowing about the lack of an ETA on a fix. Thanks!0 -
Default Wordpress 301 Redirects of JS and CSS files. Bad for SEO & How to Fix?
Hi there: We are developers with some digital marketing expertise, but a current issue has us perplexed. An outside SEO firm has asked us to clean up a large number of 301 redirects. Most of these are 'default' Wordpress behavior that relate to calling the latest version of a JS or CSS file. For instance, a JS file is called with this: https://websitexyz.com/wp-includes/js/wp-embed.min.js?ver=4.9.1 but ultimately redirects to this: https://websitexyz.com/wp-includes/js/wp-embed.min.js. We are being asked to prevent the redirect from happening by, presumably, calling the ultimate file to begin with. The issue is that, as far as we know, there's no easy way to alter WP behavior to call the ultimate file to begin with. Does anyone have any thoughts on this? Thanks.
Intermediate & Advanced SEO | | Daaveey0 -
Will disallowing URL's in the robots.txt file stop those URL's being indexed by Google
I found a lot of duplicate title tags showing in Google Webmaster Tools. When I visited the URL's that these duplicates belonged to, I found that they were just images from a gallery that we didn't particularly want Google to index. There is no benefit to the end user in these image pages being indexed in Google. Our developer has told us that these urls are created by a module and are not "real" pages in the CMS. They would like to add the following to our robots.txt file Disallow: /catalog/product/gallery/ QUESTION: If the these pages are already indexed by Google, will this adjustment to the robots.txt file help to remove the pages from the index? We don't want these pages to be found.
Intermediate & Advanced SEO | | andyheath0 -
Prevent Google from crawling Ajax
With Google figuring out how to make Ajax and JS more searchable/indexable, I am curious on thoughts or techniques to prevent this. Here's my Situation, we have a page that we do not ever want to be indexed/crawled or other. Currently we have the nofollow/noindex command, but due to technical changes for our site the method in which this information is being implemented if it is ever displayed it will not have the ability to block the content from search. It is also the decision of the business to not list the file in robots.txt due to the sensitivity of the content. Basically, this content doesn't exist unless something super important happens, and even if something super important happens, we do not want Google to know of its existence. Since the Dev team is planning on using Ajax/JS to pull in this content if the business turns it on, the concern is that it will be on the homepage and Google could index it. So the questions that I was asked; if Google can/does index, how long would that piece of content potentially appear in the SERPs? Can we block Google from caring about and indexing this section of content on the homepage? Sorry for the vagueness of this question, it's very sensitive in nature and I am trying to avoid too many specifics. I am able to discuss this in a more private way if necessary. Thanks!
Intermediate & Advanced SEO | | Shawn_Huber0 -
Multilingual Site and 301 redirection
Hey there awesome people of Moz I have this site that has many languages in it. The main language is English and my developer did the following www.example.com ( is the main site ) which redirects with a 301 to www.example.com/en if your geo location is supported by our languages then you will automatically be redirected to whatever language you have in your country but does the first language with is english have to 301 redirect to www.example.com/en ? I thought that the right way is to just leave /en at the root file. Thanks in advance
Intermediate & Advanced SEO | | Angelos_Savvaidis0 -
SEO impact difference between a URL Rewrite and 301 redirect
Hi guys and girls! Just putting a new site live, we changed the URL from one thing to another and I created a 301 file redirecting the urls like for like. The developer installing it has created a different file with columns like: RewriteRule ^page/ http://www.site/page [R=301,L] RewriteRule ^/page/ http://www.site/page [R=301,L] What's the difference? The page redirects but is there a difference between the 301 redirect and this URL rewrite in terms of SEO and link value?
Intermediate & Advanced SEO | | shloy23-2945840 -
Is it a problem to use a 301 redirect to a 404 error page, instead of serving directly a 404 page?
We are building URLs dynamically with apache rewrite.
Intermediate & Advanced SEO | | lcourse
When we detect that an URL is matching some valid patterns, we serve a script which then may detect that the combination of parameters in the URL does not exist. If this happens we produce a 301 redirect to another URL which serves a 404 error page, So my doubt is the following: Do I have to worry about not serving directly an 404, but redirecting (301) to a 404 page? Will this lead to the erroneous original URL staying longer in the google index than if I would serve directly a 404? Some context. It is a site with about 200.000 web pages and we have currently 90.000 404 errors reported in webmaster tools (even though only 600 detected last month).0 -
Are there any negative effects to using a 301 redirect from a page to another internal page?
For example, from http://www.dog.com/toys to http://www.dog.com/chew-toys. In my situation, the main purpose of the 301 redirect is to replace the page with a new internal page that has a better optimized URL. This will be executed across multiple pages (about 20). None of these pages hold any search rankings but do carry a decent amount of page authority.
Intermediate & Advanced SEO | | Visually0