How long after a URL starts showing a 404 does Google stop crawling?
-
Before hiring me to do SEO, a client re-launched their site and did not 301 the old URLs to the new. Only the home page URL stayed the same.
For a month after the re-launch, the old URLs returned a 404. For the next month, all 404 pages (basically any non-existent URL) were 301'd to the home page. Finally, 2 months after launching, they properly 301'd the old URLs to the new.
Now, the new URLs are not ranking well. I assume it's too late to realize any benefit from the 301's, just checking to see if anybody has any insight into how long Google keeps trying to crawl old/404/improperly 301'd URLs.
Thanks!
-
Thanks for the responses. I tend to agree, but my concern is the "permanent" nature of a 301. I assume they will re-crawl the old URLs and replace the previous redirects with the new ones eventually.
-
Agree. Google will follow the links. Keep checking webmaster tools and 301 the pages Google can't find. I can't see why you would not get the link value.
-
Google's crawlers follow links. As long as links exist, those links will be crawled, even if the link generates a 404 error. They could choose to follow known 404 links less frequently, but they will still make attempts to follow the link.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How long does Google take to reduce the index size?
A few months ago, we have incorporated our custom search in our website www.ergodotisi.com . We hadn't been paying a lot of attention to our webmaster analytics, to find out a few months later than the Google Index had grown from 2K- 3K pages to one million because it was crawling all combinations of search filters. We have now followed the right instructions to add noindex meta tags and blocked most search result pages from the robot.txt. We allow indexing of some main categories by setting new seo-friendly url structures. A few weeks have passed and the index size has only reduced to 700K. How long does it take before it removes most of the duplicated search result pages from the index? Is it still crawling those pages but has not fully decided to remove most of them? How bad is this for SEO?
On-Page Optimization | | cplastiras840 -
Change of URL + SEO Impact
We recently changed the URLs from our site http://www.website.com (old)
On-Page Optimization | | ederdesign
https://website.com/us/en/ (new) Our ranking is plummeting every since and I wonder if the new URL had something to do with it. Do you know if that change, could have impacted the ranking?0 -
Google Search Console issue: "This is how Googlebot saw the page" showing part of page being covered up
Hi everyone! Kind of a weird question here but I'll ask and see if anyone else has seen this: In Google Search Console when I do a fetch and render request for a specific site, the fetch and blocked resources all look A-OK. However, in the render, there's a large grey box (background of navigation) that covers up a significant amount of what is on the page. Attaching a screenshot. You can see the text start peeking out below (had to trim for confidentiality reasons). But behind that block of grey IS text. And text that apparently in the fetch part Googlebot does see and can crawl. My question: is this an issue? Should I be concerned about this visual look? Or no? Never have experienced an issue like that. I will say - trying to make a play at a featured snippet and can't seem to have Google display this page's information, despite it being the first result and the query showing a featured snippet of a result #4. I know that it isn't guaranteed for the #1 result but wonder if this has anything to do with why it isn't showing one. VmIqgFB.png
On-Page Optimization | | ChristianMKG0 -
Moz crawl
Hi, 1. Is there any way to get the moz crawl to analyse any site like the website auditor tool in seo powersuite on command. It would be good to see what keyword my competitors are using. would also like to analyse my site when i have corrected errors on command. 2. Is google adwords keyword tool still the best keyword research tool? I Don't see anything in the newbie guides to the moz pro tool regarding keyword research. Thanks! David.
On-Page Optimization | | WallerD0 -
Ok to ignore Overly-Dynamic URL from Moz crawl?
I am developing an ecommerce site, just ran it through the Moz crawl to see what's what and it has come back with a lot of issues. Most of these issues are around duplicate page titles (it is not happy with paginated titles, ie Shoes, Shoes Page 2, Shoes Page 3 etc) and it has also found a lot of Overly-Dynamic URL's. Again, these seem to be from some of the search functions and filters used Accessories&pto_sort=priceAsc&pto_page=6 other than spending a lot of time and effort trying to rewrite these urls there is little I can do about them. Should I just ignore this? I wouldn't imagine it having a massive impact on the rankings of the pages. Thanks, Carl
On-Page Optimization | | GrumpyCarl0 -
Why Does SEOMOZ Crawl show that i have 5,769 pages with Duplicate Content
Hello... I'm trying to do some analysis on my site (http://goo.gl/JgK1e) and SEOMOZ Crawl Diagnostics is telling me that I have 5,769 pages with duplicate content. Can someone, anyone, please help me understand: how does SEOMOZ determine if i have duplicate content Is it correct ? Are there really that many pages of duplicate content How do i fix this, if true <---- ** Most important ** Thanks in advance for any help!!
On-Page Optimization | | Prime850 -
Why is On-Page showing canonical wrong?
I'm trying to use the On-Page report card and it's saying that my rel=canonical is wrong, I've looked and I can't see anything wrong with it, am I missing something? The url is www.harrisonlighting.co.uk/childrens-lights.html
On-Page Optimization | | HarrisonLighting0 -
Google Reconsideration
Our site fell from grace last July and landed on page five of the Google search results for our primary keyword. For 6 months I tried a number of strategies with no results, including reconfiguring our site based on the SOEmoz on-page grading tool. More recently, after receiving your advice in a Q&A, I took down all of my paid links and submitted a reconsideration request to Google. Interestingly, 3 days later we popped up 20 spots. This left us on the top of page three. Better than page 5, but still not prime time! A few days ago (two weeks after our reconsideration request was submitted) I got a message back in my Webmaster Tools, that they had completed a review of our site - but oddly enough they provided no info on the outcome, positive or negative. And there has been no additional movement in the rankings since I received the message. Was the original 20 point jump the result of the reconsideration request, or just a coincidence? Or, is it possible that they did a review and the results will only occur later during some organic re-indexing process? What do you think?
On-Page Optimization | | JimSkychief0