How long after a URL starts showing a 404 does Google stop crawling?
-
Before hiring me to do SEO, a client re-launched their site and did not 301 the old URLs to the new. Only the home page URL stayed the same.
For a month after the re-launch, the old URLs returned a 404. For the next month, all 404 pages (basically any non-existent URL) were 301'd to the home page. Finally, 2 months after launching, they properly 301'd the old URLs to the new.
Now, the new URLs are not ranking well. I assume it's too late to realize any benefit from the 301's, just checking to see if anybody has any insight into how long Google keeps trying to crawl old/404/improperly 301'd URLs.
Thanks!
-
Thanks for the responses. I tend to agree, but my concern is the "permanent" nature of a 301. I assume they will re-crawl the old URLs and replace the previous redirects with the new ones eventually.
-
Agree. Google will follow the links. Keep checking webmaster tools and 301 the pages Google can't find. I can't see why you would not get the link value.
-
Google's crawlers follow links. As long as links exist, those links will be crawled, even if the link generates a 404 error. They could choose to follow known 404 links less frequently, but they will still make attempts to follow the link.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Is the HTML content inside an image slideshow of a website crawled by Google?
I am building a website for a client and i am in a dilemma whether to go for an image slideshow with HTML content on the slides or go for a static full size image on the homepage. My concern is that HTML content on the slideshow may not get crawled by Google and hence may not be SEO friendly.
On-Page Optimization | | aravinn0 -
My site's articles seem to never show up in Google.
This is in regards to a previous post that was answered for me:
On-Page Optimization | | Ctrl-Alt-Success
http://moz.com/community/q/my-site-s-name-not-ranking-in-google I was talking to a friend and he suggested I try to type in an article in google with the exact name followed by my site's domain name without the .com For example, I have an article entitled: "MULTITASKING IS BAD FOR YOU, MKAY?" Obviously it's a title most would not word in that way. I typed it in and followed it up with my site's domain minus .com. So "MULTITASKING IS BAD FOR YOU, MKAY? ctrl-alt-success" But I'm not even getting listed in the search. There's got to be something I'm missing. I understand backlinks are important for ranking, but when I'm trying to find an exact match along with my site's url minus the .com? I just have this strong hunch that something is awry. NOTE: It seems this is only with google. If I use Bing or Yahoo, it comes up just fine.0 -
How do I remove a Canonical URL Tag?
Some of my report cards say I have too many canonical URL tags. However, there is no information no how to delete one. Can someone give me a link or explain? Thanks.
On-Page Optimization | | dealblogger0 -
Why isn't our site being shown on the first page of Google for a query using the exact domain, when its pages are indeed indexed by Google
When I type our domain.com as a query into Google, I only see one of our pages on the homepage, and it's in 4th position. It seems though, that all pages of the site are indexed by google when I type in the query "site:domain.com". There was an issue at the site launch, where the robots.txt file was left active for around two weeks. Would this have been responsible for the fact that another domain ranks #1 when we type in our own domain? It has been around a couple of months now since the site was launched. Thanks in advance.
On-Page Optimization | | featherseo0 -
Importance of URL Structure
We are trying to restructure our onpage SEO and want to make sure we have our URLs correct. The problem is we did the URLs incorrectly in the first place and the ones we currently have are several years olds. We have some URLs such as: http://www.firebrandtraining.co.uk/courses/management/prince2.asp and
On-Page Optimization | | RobertChapman
http://www.firebrandtraining.co.uk/courses/cisco/ccna_2007.asp which are not ideal but user experience aside does it make sense for us to change the URLs and use 301 redirects to the new ones or is the damage done to our natural rankings simply not worth making the change? I have read different articles saying different things, some say that URL structure has little weight (if any weight at all) on rankings while other people seem to say it is quite important. In addition we have heard that changing the URLs with a 301 redirect will cause a large drop in ranking which will take months to recover from and contrarily that 301s are now considered "ok" by Google and we shouldn't see too much change at all in our rankings. Any advice would be much appreciated.0 -
Which Canonical URL Tag tag should we remove?
Hi guys, We are in the process of optimizing the pages of our new site. We have used the 'on page' report card feature in the Seomoz Pro Campaign analyser. On several pages we got the following result No More Than One Canonical URL Tag Number of Canonical tags <dl> <dd>2</dd> <dt>Explanation</dt> <dd>The canonical URL tag is meant to be employed only a single time on an individual URL (much like the title element or meta description). To ensure the search engines properly parse the canonical source, employ only a single version of this tag.</dd> <dt>Recommendation</dt> <dd>Remove all but a single canonical URL tag</dd> </dl> I have looked into the source code of one of the pages http://www.sabaileela.co.uk/acupuncture-london and can see that there are two "canonical" tags. Does anyone have any advise on which one I should ask the developer to remove? I am not sure how to determine the relative importance of either link.
On-Page Optimization | | brian.james0 -
Landing page too long
In my first seomoz test I have many tilte pages for products that are over 70 charachers long. The part #'s are long like 10-782-10-10-10-PPxxxx etc. All these part #'s are not my key products and I could delete or truncate. My question is if the part numbers are not that important, is it OK to leave them as is or is ranking being damaged because they exist?
On-Page Optimization | | Wales0 -
Changing page titles and google penalties?
I just recently learned that changing your page title earns you a google penalty. Unfortunately i learned this after playing around with my page titles a bit to get the most optimal page titles. Does anybody know how long this google penalty lasts? is it forever? or just temporary?
On-Page Optimization | | adriandg0