Tricky 301 question
-
A friend has relaunched a website but his web guys (he didn't consult me!) didn't do any 301s and now traffic unsurprisingly has tanked.
The old site and database no longer exists and there are now 2000+ 404's.
Any ideas how to do the 301s from old urls to new product urls WITHOUT it being a massive manual job?
-
That's my point, you only need to worry about the pages that had external links
Thanks -
Thanks
-
Pages dont just get equity from external links of course. If a category page has 10 links to it the product pages linked to on that page benefit. The wholesale drop in rankings isn't because every page had an external link to it.
-
I don't know what you mean about link equity, if there is no link pointing to the page then there is nothing lost.
As for search engines finding a lot of 404s, they will remove them from the index after a while, no problem there, you are returning the correct status code, that's what they want. This will allow them to clean up there index and stop crawling the pages. -
If the majority of URLs have no logic, then it makes things a bit tricky in regards to minimizing the amount of work.
I once had a very active and large website with about 500-1000 single lines of rewrite code (1 for each URL) in my htaccess. Surprisingly, it did not slow the server down at any noticeable rate, unless you are very sensitive to milliseconds and even then, one trial to the next could easily differ from regular internet congestion. My point is, nobody ever noticed.
Here's a few ways that I would handle this job to get through it as quickly and effortlessly as possible.
The more aggressive and time consuming approach:
I would output all the URLs that were changed from phpmyadmin or whatever mysql administration tool you might use to a spreadsheet. From that spreadsheet, I would add the original URL.
Then with the old URL (A1) and new URL (A2) I would write a formula to output the correct rewrite (A3.) Then simply copy and paste that formula down all the rows that it applies to. You might need to break up the URLs to grab the right pieces for your formula.Of course use, regex where you can, and keep your .htaccess rewrites to a minimum.
If that is still too much work, hire someone to do it through elance.com
The somewhat sloppy pace-yourself-approach:
Another approach you could take is to just monitor google webmaster tools for all the page not found errors. And once a day or once a week, grab those URLS, create the rewrite, and mark it as fixed in webmaster tools.
The reason I say this is somewhat sloppy is because, you might find that you could have used regex in a lot of instances to better handle all those missing URLs.
But it may be a good way of staying on track with google, and handling the issues only as they arise so it does not feel like such a mammoth task.
-
Thanks Alan, yes they have good external links to many pages. They retail a very niche product and have a lot of forum, review, social type links. It might be though if need be they just have to focus mostly on 301s for the pages with those links. As best practise I am in favour of 301'ing regardless of external links as the link equity gets messed up and causes ranking issues, as in this case, as well as sending a signal to the engines about the amount of wasted resource they will use crawling a site with 1000s of 404s.
-
Thanks Donna & Luis. Luis is right i'm looking for a way for this not to be a mammoth manual task for their developer.
-
Thanks, the regex is a good idea and might be part of the solution for some urls at least but there seems to be some discrepancies in logic between old and new product urls and some of the new product urls are actually still the same as the old (which of course is fine).
-
Thanks Luis, unfortunately neither 1 or 2 are ideal.
1. I don't think there is much logic in the change of url structure between old and new product urls which makes that idea impossible.
2. Thats going to be a last resort
Andy
-
do you know if they had any external links?
If they don't have external links then I would just let them 404.
some people have some wired thoughts of what 301's do. They simply redirect a request, so a request o A is told to remake the request to to B, so the crawler will follow it that way and award the pagerank to the new page with a small loss on each request.If no external links what is there to gain? don't complicate your site with unnesasary redirects, there is a small argument that the pages may have been bookmarked at old url, but I think that argument is so weak I would not bother
-
Yeah. I heard him. I guess I'm saying "probably not".
I like how you're keeping us honest though Luis. I don't like it when people respond with what they want to say rather than with an answer to the specific question.
-
Donna,
Andy has been very specific about this: "WITHOUT it being a massive manual job" hehe thanks for supporting my answer.
Luis
-
It really depends on the nature, link and traffic patterns of your site Andy. If the vast majority of those 2,000+ 404's are coming from pages that should never have been indexed in the first place, you can probably get away with Luis's 2nd suggestion. If they're differentiated, valuable, and show evidence of incoming links and traffic, you've got some work ahead of you.
You might be able to streamline the process by inventorying and grouping like pages, then doing group redirects. But I suggest you do some analysis first to determine whether the effort is warranted.
-
2000+ is a lot of URLs to work through. But you can most likely get through them quickly with a few good regular expression 301 redirects in your .htaccess
If you have a pretty consistent form from the old url to the new one, this will be a piece of cake.
ex:
old URL: this/was/coolnew URL: this/is/cool
However, if there is really no rhyme and reason to the newly formed URLs, this could end up taking a considerate amount of time.
I would look into writing 301 redirects with regular expressions in .htaccess (I'm assuming your server is and uses .htaccess)
There are a number of resources for doing this, and even one here at moz.com
https://moz.com/learn/seo/redirection -
Hello Andy,
1. Try this: http://webdesign.about.com/od/htaccess/ht/redirect-an-entire-site-using-htaccess.htm
2. Second/faster solution. You could add this line of code to your .htacess file (and all the current "404's users" will go to the homepage):
ErrorDocument 404 /
But pay attention... 404's are perfectly normal if the page no longer exists, for user experience you should only ever use a 301 redirect if the page that no longer exists is going to a equal page.. i.e about cars to cars, about rabbits to rabbits. Maybe the only solution is creating a 404 specific landing page for this (with links to different sections of your site)
Hope this helps,
Luis
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Question about related topics
I know that in order to rank on any keyword I need to talk about different "concepts / topics " everyone has a different word for it but let say I need to talk about multiple subject to make it simple. My question is how to find those subjects ... in some industry it is pretty straight forward you go to related search or some keyword tool such as Moz and you find what you need. Example : If "Title tag" is my main topic the subtopic that I find and that I need to cover on the same page are "title tag length , title tag checker, mobile title tag, title tag example etc..." On my keywords such as " Alsace bike tours" all I find in related searches and using all the tools out there such as Moz keyword research explorer is "Alsace cycling vacations " ""Cycling Colmar" "Alsace bike trip " etc... not really anything exiting , it means the same thing and it is just variations of the keyword. I have used other tools such as Marketmuse and they give me related topic such as "Strasbourg" "Colmar" "half timbered houses" "Alsace wine" and I am not sure it is any better... because to cover those I have no other solution that doing definitions... or describe those in details which is probably not what someone typing "Alsace bike tour" is looking for. I have the feeling that all those tools are great for keywords like "content marketing" or "title tag" with a lot of requests but they fail for everything else. Can someone give me an insight on how they do to write on multiple topic where they are in my situation and based on the example I gave which topic they would cover and based on my example.. Thank you,
Intermediate & Advanced SEO | | seoanalytics1 -
Question on AMP
I'd like to utilize AMP for faster loading for one of my clients. However, it is essential that this client have chat. My developer is having trouble incorporating chat with AMP, and he claims that it isn't possible to integrate the two. Can anyone advise me as to whether this is accurate? If it is true that AMP and chat aren't compatible, are there any solutions to this issue? I'd appreciate any leads on this. Thanks!
Intermediate & Advanced SEO | | Joseph-Green-SEO0 -
Ecommerce catalog update: 301 redirects?
Hello mozers, We run an ecommerce store and are planning a massive catalog update this month. Essentially, 100% of our product listings will be deleted, and an all new catalog will be uploaded. The new catalog contains mostly new products, however there are some products that already existing in the old catalog as well. The new catalog has a bunch of improvements to the product pages, included optimized meta titles and descriptions, multiple language, optimized URLs and more. My question is the following: When we delete the existing catalog, all indexed URLs will return 404 errors. Setting up 301 redirects from old to new products (for products which existing previously) is not feasible given the number of products. Also, many products are simply being remove entirely. So should we go ahead and delete all products, upload the new catalog, update the sitemap, resubmit it for crawling, and live with a bunch of 404 errors until these URLs get dropped from Google? The alternative I see is setting 301 redirects to the home page, but I am not sure this would be correct use of 301 redirects. Thanks for your input.
Intermediate & Advanced SEO | | yacpro130 -
E-Commerce Panda Question
I'm torn. Many of our 'niche' ecommerce products rank well, however I'm concerned that duplicate content is negatively effecting our overall rankings via Panda Algo. Here is an example that can be found through quite a few products on the site. This sub-category page (http://www.ledsupply.com/buckblock-constant-current-led-drivers) in our 'led drivers' --> 'luxdrive drivers' section has three products that are virtually identical with much of the same content on each page, except for their 'output current' - sort of like a shirt selling in different size attributes: S, M, L and XL. I could realistically condense 44 product pages (similar to example above) down to 13 within this sub-category section alone (http://www.ledsupply.com/luxdrive-constant-current-led-drivers). Again, we sell many of these products and rank ok for them, but given the outline for how Panda works I believe this structure could be compromising our overall Panda 'quality score', consequently keeping our traffic from increasing. Has anyone had similar issues and found that its worth the risk to condense product pages by adding attributes? If so, do I make the new pages and just 301 all the old URLs or is there a better way?
Intermediate & Advanced SEO | | saultienut0 -
301 Redirecting Multiple Domains
I have several complete websites with blogs setup for different geo locations and was considering forwarding them all to one domain directly would greatly benefit ranking. The blogs are all linked together and that is where most of the links come from. Would I benefit in 301 Redirecting the domains?
Intermediate & Advanced SEO | | WindshieldGuy-2762210 -
Are videos content to Google bot? and other questions.
It seems as though my site has been hit, possibly because of above the fold adverts or lack of content above the fold, so I have a number of questions regarding this. 1. Are videos regarded as content by Google Bot? 2. If three adverts are placed above the fold with text content clearly readable. Will these three adverts still affect my search engine rankings? 3. Is it better to put text before the video and have the video placed a bit lower? 4. I have a number of pages that have video but no text, could these pages combine to decrease the value of my best landing pages? thanks 😄
Intermediate & Advanced SEO | | phoenixcg0 -
Duplicate Content Question
Brief question - SEOMOZ is teling me that i have duplicate content on the following two pages http://www.passportsandvisas.com/visas/ and http://www.passportsandvisas.com/visas/index.asp The default page for the /visas/ directory is index.asp - so it effectively the same page - but apparently SEOMOZ and more importantly Google, etc treat these as two different pages. I read about 301 redirects etc, but in this case there aren't two physical HTML pages - so how do I fix this?
Intermediate & Advanced SEO | | santiago230 -
Duplicate content question? thanks
Hi, Im my time as an SEO I have never come across the following two scenarios, I am an advocate of using unique content, therefore always suggest and in cases demand that all content is written or re-written. This is the scenarios I am facing right now. For Example we have www.abc.com (has over 200 original recipes) and then we have www.xyz.com with the recipes but they are translated into another language as they are targeting different audiences, will Google penalize for duplicate content? The other issue is that the client got the recipes from www.abc.com (that have been translated) and use them in www.xyz.com aswell, both sites owned by the same company so its not pleagurism they have legal rights but I am not sure how Google will see it and if it will penalize the sites. Thanks!
Intermediate & Advanced SEO | | M_81