How to handle potentially thousands (50k+) of 301 redirects following a major site replacement
-
We are looking for the very best way of handling potentially thousands (50k+) of 301 redirects following
a major site replacement and I mean total replacement.Things you should know
Existing domain has 17 years history with Google but rankings have suffered over the past year and yes we know why. (and the bitch is we paid a good sized SEO company for that ineffective and destructive work)
The URL structure of the new site is completely different and SEO friendly URL's rule. This means that there will be many thousands of historical URL's (mainly dynamic ones) that will attract 404 errors as they will not exist anymore. Most are product profile pages and the God Google has indexed them all. There are also many links to them out there.
The new site is fully SEO optimised and is passing all tests so far - however there is a way to go yet.So here are my thoughts on the possible ways of meeting our need,
1: Create 301 redirects for each an every page in the .htaccess file that would be one huge .htaccess file 50,000 lines plus - I am worried about effect on site speed.
2: Create 301 redirects for each and every unused folder, and wildcard the file names, this would be a single redirect for each file in each folder to a single redirect page
so the 404 issue is overcome but the user doesn't open the precise page they are after.
3: Write some code to create a hard copy 301 index.php file for each and every folder that is to be replaced.
4: Write code to create a hard copy 301 .php file for each and every page that is to be replaced.
5: We could just let the pages all die and list them with Google to advise of their death.
6: We could have the redirect managed by a database rather than .htaccess or single redirect files. Probably the most challenging thing will be to load the data in the first place, but I assume this could be done programatically - especially if the new URL can be inferred from the old.Many be I am missing another, simpler approach - please discuss
-
Sorry to hear of your woes.
Depending on the structure of the URLS you could create some simple pattern matches rules within .htaccess? If you could a few dozen rules could handle many thousands of redirects. If there isn't any easily identifiable pattern to match then a DB will, indeed, be your best option.
One of the web devs I used to work with (who was considerably smarter than me) faced a similar issue (with a 'mere' 10k+ redirects) and used some Ruby on Rails middleware as a redirector: This may have been the solution he used:
https://github.com/vigetlabs/redirector
I hope that helps.
I hope you're able to get this sorted without too much pain. Good Luck!
-
Thank for the very quick response - you have picked my favourite solution. It will be interesting to hear other views and comments.
-
Hi,
1. Usually won't work and with 50k extra rules in your htaccess file it will for sure slow down the site as for every request to your server it has to go through the htaccess file.
For now I would recommend going with 6. with the information that you've provided. By doing it like this you can do a very quick check on your database and also in the request have the user send to the right page.
Martijn.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
If UGC on my site also exists elsewhere, is that bad? How should I properly handle it?
I work for a reviews site, and some of the reviews that get published on our website also get published on other reviews websites. It's exact duplicate content -- all user generated. The reviews themselves are all no-indexed; followed, and the pages where they live are only manually indexed if the reviews aren't duplicate. We leave all pages with reviews that live elsewhere on the web nofollowed. Is this how we should properly handle it? Or would it be OK to follow these pages regardless of the fact that technically, there's exact duplicate UGC elsewhere?
Intermediate & Advanced SEO | | dunklea0 -
301 or Canonical - Ecommerce Site Question
We are making a change to our Navigation and this includes having to change the URL structure of a few pages of our site. Due to issues with the CMS (that are out of my control) we are unable to keep the current URL structure of two of our highest ranking pages. Our site is an E-commerce Site The Structure is changing from..... www.domain.com/page/highrankingpage <----OLD PAGE RANKED WELL to www.domain.com/category/highrankingpage <----NEW PAGE Generally I would have 301 'd this page but I found out that our Tech team added a Canonical to this page instead....(showing the high ranking page to the Search Engines) and on our site the visitors are able to browse the website getting the new page. BOTH PAGES ARE BASICALLY IDENTICAL (Same Content) http://searchenginewatch.com/sew/how-to/2288690/how-and-when-to-use-301-redirects-vs-canonical# Thoughts?
Intermediate & Advanced SEO | | CMcMullen0 -
301 Redirection
Hi there guys, I have a question about redirection. My boss has just bought a new domain name and he wants it to redirect to our current site when looking for specific products. www.example.com is our current website www.productname.com is the new domain So the new domain would be redirected to example.com. Would that be considered against Google Policies? Thanks
Intermediate & Advanced SEO | | PremioOscar0 -
301 Redirect from unused domain
Hi All First question here so go easy.. I have a property site which is working well so far considering it;s early days, unfortunately some of my earlier efforts did not go so well and one in particular I pretty much destroyed in my attempts to improve the site SEO. Lucky enough my SEO skills have improved quite a bit lately, largely thanks to the great tools, tutorials and experts here at Moz 🙂 My question is whether I can use a 301 redirect to pass the domain authority and any link equity from an unused site to the one that ive done a better job on? it would seem a little sketchy to me and I would prefer not to get slapped and penalized "again" for doing something dodgy... Thanks everyone and thanks for all the help over the last 6 months or so.. Wes Dunn
Intermediate & Advanced SEO | | wesdunn19771 -
Does this require site-wide 301 redirects?
I have an old site that we are re-building, and also moving form Yahoo Stores to Big Commerce. yahoo uses site.com/page.html and BC uses site.com/page. Is there any SEO benefit to keeping the old .html format? some of the pages on the old site have no links to them from external sites. Do they even need re-directs, or should I just let Google find the new page equivalents when they crawl the new version of the site? While some of the old pages (primarily product pages) have OK urls, others have obscure product numbers as the URL. Obviously the latter need re-directing to a more relevant page, but what about situations like this:
Intermediate & Advanced SEO | | Grabapple
_/accessory-product.html _ > product-accessory
In this example, the existing URL is fine, except for the .html extention. If I just used the old URL, would having a mix of /sample.html and /sample pages hurt me? Thanks in advance for your help and input! Dave0 -
How to set up 301 redirect for URL with question mark
I have encountered some issue with 301 redirect and htaccess file. I need to redirect the following url: http://www.domain.com/?specifications=colours/page/3 to: http://www.domain.com/colours The 301 redirect command I wrote in htaccess file is as follow: Redirect 301 /?specifications=colours/page/3 http://www.domain.com/colours And it doesn't work at the moment. What is the correct way to set up 301 redirect here? Your help will be sincerely appreciated!
Intermediate & Advanced SEO | | robotseo0 -
Do my redirects on my homepage need to be 301?
Our domain name is something like www.I-am-cool.com but most people just type in iamcool.com After doing some research I found that those are 302 redirects and I think they should be 301. If I am correct do I need to redirect www.iamcool.com and iamcool.com or just one or the other?
Intermediate & Advanced SEO | | EcommerceSite0 -
Have a problem with our home page. Is temporary 301 redirect an option?
Hey Mozers, I discovered this morning that the home page for my website is rendering fine in Chrome and Firefox, but very poorly in IE. My analytics show that over 50% of my visitors are using IE. As a result of the problem, IE has a bounce rate 32% higher than other browsers. I'm not a web developer and I'm fairly new to SEO, so I'm guessing that it's going to take me at least a couple days to get it fixed. In the meantime, I was considering doing a 301 redirect from the home page to the largest category page in hopes of keeping some of the IE users from bouncing while I get the home page sorted out. Would there be any long term negative effects from this once I get the page sorted out and take the 301 off it? Are there any other solutions that would be better? Thanks for the help!
Intermediate & Advanced SEO | | matthewbyers0