Does Google make continued attempts to crawl an old page one it has followed a 301 to the new page?
-
I am curious about this for a couple of reasons.
We have all dealt with a site who switched platforms and didn't plan properly and now have 1,000's of crawl errors. Many of the developers I have talked to have stated very clearly that the HTacccess file should not be used for 1,000's of singe redirects. I figured If I only needed them in their temporarily it wouldn't be an issue. I am curious if once Google follows a 301 from an old page to a new page, will they stop crawling the old page?
-
It will keep on crawling. I have seen 6 months to 1 year later they come back. Especially if there are links from other websites pointing to your old URLs.
Why they do repeat crawls on 301s and 404s for that matter is that they know how much the web changes and assume that there may have been a mistake and so want to double check. Also, you have to consider, they will not recrawl 1000s of pages all at once. They will probably pick up the most popular URLs first as they crawl them more often and then the less popular pages later.
IMHO. You will need to find a long term solution in place.
Here is the why you dont want to use htaccess
http://stackoverflow.com/questions/209209/htaccess-redirect-performance
One suggestion for Apache - RewriteMap
http://httpd.apache.org/docs/2.0/mod/mod_rewrite.html#rewritemap
That or all the developers who are telling you not to use .htaccess should have at least 2 solutions to use instead that should work. Go back to them to solve the issue.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to increase Page authority of old blog posts
Hi, How can I increase the page authority of old blog posts? There are many posts that are ranking well (Page 1 lower or Page 2) - but I want to make them rank higher by making the post more usable, better UI, design, content relaunches etc - these all would inherently mean improving Page authority also eventually. What are some concrete steps I can take to improve page authority of blog pages?
Intermediate & Advanced SEO | | pks3331 -
301 process, migration to new domain
Hi all! We have an old site wordpress based, with great ranking and PR 7, called www.europe-internship.com which is going to be migrated into our new Django site www.eurasmus.com (specifically eurasmus.com/en/europe-internships)
Intermediate & Advanced SEO | | Eurasmus.com
The new one is a much more advanced version that we will keep developing. We have been migrating the information already but we are planning to apply the 301s in the next weeks to start passing the SEO value to our new site and traffic. We have all the url structures and everything checked and technically we are ready for it.
Therefore, we are almost ready. I have 2 questions: The new site includes more services, like accommodation, information...not only internships. Therefore, should we point the most relevant urls from our previous site to our home to share the value or just to the internships section? I am afraid that if the bounce rate goes higher from the 301 we could loose some value... 2)Should we point all the urls at the same time to the new site? Home, vacancies, blog pages, etc... or start gradually doing it to see how it goes till we make it to all the pages including the home? The old site still makes some money and I am not sure how quick will be to pass the SEO value, so in the way we may loose few thousand euros...We understand that, but we want to check what would be the best in your opinion. Let me know what you think and your opinion! Thank you in advance!0 -
SEO structure question: Better to add similar (but distinct) content to multiple unique pages or make one unique page?
Not sure which approach would be more SEO ranking friendly? As we are a music store, we do instrument repairs on all instruments. Currently, I don't have much of any content about our repairs on our website... so I'm considering a couple different approaches of adding this content: Let's take Trumpet Repair for example: 1. I can auto write to the HTML body (say, at the end of the body) of our 20 Trumpets (each having their own page) we have for sale on our site, the verbiage of all repairs, services, rates, and other repair related detail. In my mind, the effect of this may be that: This added information does uniquely pertain to Trumpets only (excludes all other instrument repair info), which Google likes... but it would be duplicate Trumpet repair information over 20 pages.... which Google may not like? 2. Or I could auto write the repair details to the Trumpet's Category Page - either in the Body, Header, or Footer. This definitely reduces the redundancy of the repeating Trumpet repair info per Trumpet page, but it also reduces each Trumpet pages content depth... so I'm not sure which out weighs the other? 3. Write it to both category page & individual pages? Possibly valuable because the information is anchoring all around itself and supporting... or is that super duplication? 4. Of course, create a category dedicated to repairs then add a subcategory for each instrument and have the repair info there be completely unique to that page...- then in the body of each 20 Trumpets, tag an internal link to Trumpet Repair? Any suggestions greatly appreciated? Thanks, Kevin
Intermediate & Advanced SEO | | Kevin_McLeish0 -
Combining two pages into one
I've got two pages I'd like to combine into one. These two URLs: widgets.com/about_us.php & widgets.com/contact_us.php would become: widgets.com/about-us Two questions... 1. Is it okay to combine the content into one page and setup 301 redirects for both pages (without losing page rank)? 2. Do I keep the original pages in my sitemap.xml file or just take them out after I setup the redirect? Thanks in advance!
Intermediate & Advanced SEO | | seo-hunter0 -
Splitting one Website into 2 Different New Websites with 301 redirects, help?
Here's the deal. My website stbands.com does fairly well. The only issue it is facing a long term branding crisis. It sells custom products and sporting goods. We decided that we want to make a sporting goods website for the retail stuff and then a custom site only focusing on the custom stuff. One website transformed and broken into 2 new ones, with two new brand names. The way we are thinking about doing this is doing a lot of 301 redirects, but what do we do with the homepage (stbands.com) and what is the best practice to make sure we don't lose traffic to the categories, etc.? Which new website do we 301 the homepage to? It's rough because for some keywords we rank 3 or 4 times on the first page. Scary times, but something must be done for the long term. Any advise is greatly appreciated. Thank you in advance. We are set for a busy next few months 🙂
Intermediate & Advanced SEO | | Hyrule0 -
Why isnt my crawl results showing a 301 redirect even though I have a 301 rewrite in my .htaccess file?
Ive searched the previous Q&A's & cant find an answer so I;ll ask it here 🙂 crawling my site shows isnt the 301 redirect that i have from my non www to my www domainIts only showing all the results for my www subdomain.As i'm new to SEO & SeoMoz I dont fully understand. Any help would be greatly appreciated because my site is like 2 & a half years old & i'm trying to learn seo so I can rank higher in the serp's. Thanks
Intermediate & Advanced SEO | | PCTechGuy20120 -
How to make Google forget my pages ?
Hello all ! I've decided to delete many pages from my website wich had poor content. I've made a php 301 redirect from all these old pages to a unique page (not the home page, a deep page). My problem is that this modification has been made a week ago and my position in the SERPs have crashed down... What can I do ? I believe that I'll get up again when Google will see that these pages don't exist anymore but it could take a long time 😞 (these page are in the Google cache with a date older than my modification's date) I've read somewhere that I should put a link to the destination page (where old pages are 301 redirected) but I don't understand how it could help... Can someone help me ? Tell me what I've done wrong... These pages were very poor and I've deleted them in order to boost the global quality of my site... It should help me in the SERPs, not penalize me...
Intermediate & Advanced SEO | | B-CITY0 -
Rankings for Home vs. Internal Pages - Potential 301?
Hi everyone: A site I'm working with until recently was ranking page 1 for its primary keyword. For the last month, they've dropped to page 4. One thing we've noticed is that the page that is ranking is an internal page (http://www.example.com/keyword-string) and at this point, everything ranking above us is ranking based on the root domain (http://www.competitor.com). We've eliminated Panda, penalties, and any other obvious causes for the drop in rankings. We have similar or better page rank, external links, domain trust, etc. in comparison to the sites still ranking page 1. We think this may be part of our problem. Has anyone else dealt with this? What did you do to change it and how did it work? We're considering eliminating the existing internal page and 301'ing to the home page. The keyword in question is the core of the business, so this is a natural change, but we're loath to lose years of investment in promoting the internal page. Also, the site was originally optimized with the primary keyword throughout (appears in META tags, headers on multiple pages). How important is it to clear these out to make Google see the home page as most relevant? Thanks!!
Intermediate & Advanced SEO | | kdcomms0