How to handle potentially thousands (50k+) of 301 redirects following a major site replacement
-
We are looking for the very best way of handling potentially thousands (50k+) of 301 redirects following
a major site replacement and I mean total replacement.Things you should know
Existing domain has 17 years history with Google but rankings have suffered over the past year and yes we know why. (and the bitch is we paid a good sized SEO company for that ineffective and destructive work)
The URL structure of the new site is completely different and SEO friendly URL's rule. This means that there will be many thousands of historical URL's (mainly dynamic ones) that will attract 404 errors as they will not exist anymore. Most are product profile pages and the God Google has indexed them all. There are also many links to them out there.
The new site is fully SEO optimised and is passing all tests so far - however there is a way to go yet.So here are my thoughts on the possible ways of meeting our need,
1: Create 301 redirects for each an every page in the .htaccess file that would be one huge .htaccess file 50,000 lines plus - I am worried about effect on site speed.
2: Create 301 redirects for each and every unused folder, and wildcard the file names, this would be a single redirect for each file in each folder to a single redirect page
so the 404 issue is overcome but the user doesn't open the precise page they are after.
3: Write some code to create a hard copy 301 index.php file for each and every folder that is to be replaced.
4: Write code to create a hard copy 301 .php file for each and every page that is to be replaced.
5: We could just let the pages all die and list them with Google to advise of their death.
6: We could have the redirect managed by a database rather than .htaccess or single redirect files. Probably the most challenging thing will be to load the data in the first place, but I assume this could be done programatically - especially if the new URL can be inferred from the old.Many be I am missing another, simpler approach - please discuss
-
Sorry to hear of your woes.
Depending on the structure of the URLS you could create some simple pattern matches rules within .htaccess? If you could a few dozen rules could handle many thousands of redirects. If there isn't any easily identifiable pattern to match then a DB will, indeed, be your best option.
One of the web devs I used to work with (who was considerably smarter than me) faced a similar issue (with a 'mere' 10k+ redirects) and used some Ruby on Rails middleware as a redirector: This may have been the solution he used:
https://github.com/vigetlabs/redirector
I hope that helps.
I hope you're able to get this sorted without too much pain. Good Luck!
-
Thank for the very quick response - you have picked my favourite solution. It will be interesting to hear other views and comments.
-
Hi,
1. Usually won't work and with 50k extra rules in your htaccess file it will for sure slow down the site as for every request to your server it has to go through the htaccess file.
For now I would recommend going with 6. with the information that you've provided. By doing it like this you can do a very quick check on your database and also in the request have the user send to the right page.
Martijn.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Can I undo a 301 redirect? Will it penalize my ranking?
I'm in charge of building a website for a company that made a mess. They own domain xxx.it (xxx is not the real domain, just a placeholder). Some years ago they 301 redirected xxx.it to xxx.com (they just changed TLDs). Last year they 301 redirected xxx.com to yyy.com (so, they actually changed domains). Now, after 13 months, the company failed and the new leadership wants me to undo everything and 301 redirect from yyy.com to xxx.it. So: 301 redirect is permanent. So, conceptually it's wrong to undo it. What happens if I undo it? Will my ranking be penalized, even if a significant amount of time has passed (13 months)? Will crawlers detect a loop (even if i remove any 301 redirect from xxx.it and theorically break the loop)? Here is the potential loop: xxx.it -> xxx.com -> yyy.com -> xxx.it -> etc... All of the articles I found on the web are quite old and not clear about this topic, that's why I'm asking this question.
Intermediate & Advanced SEO | | naska19900 -
301 from old site to new one , Should I point to home page or sub category page ?
Hey Seo Experts, I have a small website ranking for few terms like cabinets sale, buy etc . However what i have now decided is to launch a New website with more different products like living room furniture, wardrobes etc . Out of all these categories on new website Cabinets is one of the SubCategory . Now I do not want to have 2 websites . So wanted to 301 from small cabinets website to newly created website. Some of the doubts I have at the moment is ? 1 Should I REDIRECT 301 to sub category (i,e cabinets) which is purely related to Cabinets or Do a Redirect to HOME PAGE . As I also need more Authority to home page as well , as this is relatively new website ? 2 Second question related to this. If you have multiple sub domains does it divide the total authority & TF.Or it is just Ok to have multiple Sub domains if needed ? Any advice appreciated !! Thanks .
Intermediate & Advanced SEO | | aus00070 -
301 redirect
Hi there, I have some good links pointing to one of my web pages at the moment, however we are just about to launch a new design with new URL structure and I am clear that I need to do a 301 redirect on the URL to the new URL. However, do I keep the old URL live forever? or can I remove it after a while? Kind Regards
Intermediate & Advanced SEO | | Paul780 -
How many pages to 301 Redirect
Hi Mozzers, My site has 11,200 pages indexed in Google and I'm looking to remove some of the lesser content which should probably have been picked up by Panda. However these pages work out to about 1,100 in total and I'm not sure whether to remove these bit by bit or just do it in one fell swoop? Does Google not like a site's indexed pages fluctuating too quickly? Are there any other considerations I should be aware of? Thanks!
Intermediate & Advanced SEO | | panini0 -
Is it ok to use both 301 redirect and rel="canonical' at the same time?
Hi everyone, I'm sorry if this has been asked before. I just wasn't able to find a response in previous questions. To fix the problems in our website regarding duplication I have the possibility to set up 301's and, at the same time, modify our CMS so that it automatically sets a rel="canonical" tag for every page that is generated. Would it be a problem to have both methods set up? Is it a problem to have a on a page that is redirecting to another one? Is it advisable to have a rel="canonical" tag on every single page? Thanks for reading!
Intermediate & Advanced SEO | | SDLOnlineChannel0 -
Choose of destination for a 301 redirection
Hi, I had a website paris-football.com which ranked quite well on specific request as 'paris football" and "paris foot". I decided 2 months ago to stop this website as I had no time to update it and it was quite rubish in terms of content and make a redirection to a better quality website. I decided to redirect to the deep url http://www.sportytrader.com/paris-foot.php . The destination Url has not beneft from the redirection and has even seen its rankings drop since the redirection. do you think that it would have been better to redirect to the Home Page http://www.sportytrader.com ? Do you think that I can still change the destination url ? Thanks a lot for your help,
Intermediate & Advanced SEO | | jarnac0 -
Probelms w/ 301 Redirects
I am trying to 301 redirect my old site to the new site (under the same domain. Most of the 404 urls are 301 redirects from 404 pages but they are still able to redirect. I need help understanding why I cannot redirect half of my list. <colgroup><col width="636"> <col width="342"> <col width="64"></colgroup>
Intermediate & Advanced SEO | | Melia
| OLD URL | 301 To: | Status |
| http://www.meliacaribetropical.com/en/groups.html | http://www.meliacaribetropical.com/en/groups.html | done |
| http://www.meliacaribetropical.com/entertainment/ | http://www.meliacaribetropical.com/en/services.html | done |
| http://www.meliacaribetropical.com/es/all_inclusive.html | http://www.meliacaribetropical.com/es/index.html | done |
| http://www.meliacaribetropical.com/es/galeria/ | http://www.meliacaribetropical.com/es/visor.html | done |
| http://www.meliacaribetropical.com/es/los-picapiedra/ | http://www.meliacaribetropical.com/es/index.html | done |
| http://www.meliacaribetropical.com/es/outside-us-telephone-listing.php | http://www.meliacaribetropical.com/en/index.html | done |
| http://www.meliacaribetropical.com/es/restaurantes/ | http://www.meliacaribetropical.com/es/gastronomia.html | done |
| http://www.meliacaribetropical.com/es/reuniones-eventos/ | http://www.meliacaribetropical.com/es/grupos.html | done |
| http://www.meliacaribetropical.com/es/spa/ | http://www.meliacaribetropical.com/es/servicios.html | done |
| http://www.meliacaribetropical.com/es/terms-condition.php | http://www.meliacaribetropical.com/es/index.html | done |
| http://www.meliacaribetropical.com/gallery/beach.html | http://www.meliacaribetropical.com/en/index.html | done |
| http://www.meliacaribetropical.com/gallery/dining.html | http://www.meliacaribetropical.com/en/gastronomy.html | done |
| http://www.meliacaribetropical.com/gallery/pools.html | http://www.meliacaribetropical.com/en/index.html | done |
| http://www.meliacaribetropical.com/lowest-rate.php | http://www.meliacaribetropical.com/en/index.html | done |
| http://www.meliacaribetropical.com/press/ | http://www.meliacaribetropical.com/en/index.html | done |
| http://www.meliacaribetropical.com/spanish/ | http://www.meliacaribetropical.com/es/index.html | done |
| http://www.meliacaribetropical.com/spanish/services/ | http://www.meliacaribetropical.com/en/services.html | done |
| http://www.meliacaribetropical.com/spanish/services/ | http://www.meliacaribetropical.com/es/servicios.html | done |
| http://www.meliacaribetropical.com/terms-condition.php | http://www.meliacaribetropical.com/en/index.html | done |
| http://www.meliacaribetropical.com/es/galeria/piscinas.html | http://www.meliacaribetropical.com/es/index.html |
|
| http://www.meliacaribetropical.com/es/galeria/playa.html | http://www.meliacaribetropical.com/es/index.html |
|
| http://www.meliacaribetropical.com/es/galeria/restaurantes.html | http://www.meliacaribetropical.com/es/gastronomia.html |
|
| http://www.meliacaribetropical.com/es/prensa/ | http://www.meliacaribetropical.com/es/index.html |
|
| http://www.meliacaribetropical.com/es/prensa/family-facilities-amenities.html | http://www.meliacaribetropical.com/es/index.html |
|
| http://www.meliacaribetropical.com/es/prensa/melia-caribe-tropical-announces-fall-promotion.html | http://www.meliacaribetropical.com/es/index.html |
|
| http://www.meliacaribetropical.com/es/prensa/melia-international-brand-overhaul.html | http://www.meliacaribetropical.com/es/index.html |
|
| http://www.meliacaribetropical.com/press/family-facilities-amenities.html | http://www.meliacaribetropical.com/en/index.html |
|
| http://www.meliacaribetropical.com/press/melia-caribe-tropical-announces-fall-promotion.html | http://www.meliacaribetropical.com/en/index.html |
|
| http://www.meliacaribetropical.com/press/melia-international-brand-overhaul.html | http://www.meliacaribetropical.com/en/index.html |
|
| http://www.meliacaribetropical.com/spanish/accommodations/ | http://www.meliacaribetropical.com/es/habitaciones.html |
|
| http://www.meliacaribetropical.com/spanish/dining/ | http://www.meliacaribetropical.com/es/gastronomia.html |
|
| http://www.meliacaribetropical.com/spanish/entertainment/ | http://www.meliacaribetropical.com/es/servicios.html |
|
| http://www.meliacaribetropical.com/spanish/events/ | http://www.meliacaribetropical.com/es/index.html |
|
| http://www.meliacaribetropical.com/spanish/flintstones/ | http://www.meliacaribetropical.com/es/index.html |
|
| http://www.meliacaribetropical.com/spanish/gallery/ | http://www.meliacaribetropical.com/es/visor.html |
|
| http://www.meliacaribetropical.com/spanish/gallery/beach.html | http://www.meliacaribetropical.com/es/index.html |
|
| http://www.meliacaribetropical.com/spanish/gallery/dining.html | http://www.meliacaribetropical.com/es/gastronomia.html |
|
| http://www.meliacaribetropical.com/spanish/gallery/pools.html | http://www.meliacaribetropical.com/es/index.html |
|
| http://www.meliacaribetropical.com/spanish/press/ | http://www.meliacaribetropical.com/es/index.html |
|
| http://www.meliacaribetropical.com/spanish/press/melia-caribe-tropical-announces-fall-promotion.html | http://www.meliacaribetropical.com/es/index.html |
|
| http://www.meliacaribetropical.com/spanish/press/melia-international-brand-overhaul.html | http://www.meliacaribetropical.com/es/index.html |
|
| http://www.meliacaribetropical.com/spanish/royal/ | http://www.meliacaribetropical.com/es/servicio-real.html |
|
| http://www.meliacaribetropical.com/spanish/sitemap.html | http://www.meliacaribetropical.com/es/index.html |
|
| http://www.meliacaribetropical.com/spanish/spa/ | http://www.meliacaribetropical.com/es/servicios.html |
| gastronomia.html0 -
301 Redirect All Url's - WWW -> HTTP
Hi guys, This is part 2 of a question I asked before which got partially answered; I clicked question answered before I realized it only fixed part of the problem so I think I have to post a new question now. I have an apache server I believe on Host Gator. What I want to do is redirect every URL to it's corresponding alternative (www redirects to http). So for example if someone typed in www.mysite.com/page1 it would take them to http://mysite.com/page1 Here is a code that has made all of my site's links go from WWW to HTTP which is great, but the problem is still if you try to access the WWW version by typing it, it still works and I need it to redirect. It's important because Google has been indexing SOME of the URL's as http and some as WWW and my site was just HTTP for a long time until I made the mistake of switching it now I'm having a problem with duplicate content and such. Updated it in Webmaster Tools but I need to do this regardless for other SE's. Thanks a ton! RewriteEngine On RewriteBase / RewriteCond %{HTTP_HOST} ^www.yourdomain.com [NC] RewriteRule ^(.*)$ http://yourdomain.com/$1 [L,R=301]
Intermediate & Advanced SEO | | DustinX0