Tricky 301 question
-
A friend has relaunched a website but his web guys (he didn't consult me!) didn't do any 301s and now traffic unsurprisingly has tanked.
The old site and database no longer exists and there are now 2000+ 404's.
Any ideas how to do the 301s from old urls to new product urls WITHOUT it being a massive manual job?
-
That's my point, you only need to worry about the pages that had external links
Thanks -
Thanks
-
Pages dont just get equity from external links of course. If a category page has 10 links to it the product pages linked to on that page benefit. The wholesale drop in rankings isn't because every page had an external link to it.
-
I don't know what you mean about link equity, if there is no link pointing to the page then there is nothing lost.
As for search engines finding a lot of 404s, they will remove them from the index after a while, no problem there, you are returning the correct status code, that's what they want. This will allow them to clean up there index and stop crawling the pages. -
If the majority of URLs have no logic, then it makes things a bit tricky in regards to minimizing the amount of work.
I once had a very active and large website with about 500-1000 single lines of rewrite code (1 for each URL) in my htaccess. Surprisingly, it did not slow the server down at any noticeable rate, unless you are very sensitive to milliseconds and even then, one trial to the next could easily differ from regular internet congestion. My point is, nobody ever noticed.
Here's a few ways that I would handle this job to get through it as quickly and effortlessly as possible.
The more aggressive and time consuming approach:
I would output all the URLs that were changed from phpmyadmin or whatever mysql administration tool you might use to a spreadsheet. From that spreadsheet, I would add the original URL.
Then with the old URL (A1) and new URL (A2) I would write a formula to output the correct rewrite (A3.) Then simply copy and paste that formula down all the rows that it applies to. You might need to break up the URLs to grab the right pieces for your formula.Of course use, regex where you can, and keep your .htaccess rewrites to a minimum.
If that is still too much work, hire someone to do it through elance.com
The somewhat sloppy pace-yourself-approach:
Another approach you could take is to just monitor google webmaster tools for all the page not found errors. And once a day or once a week, grab those URLS, create the rewrite, and mark it as fixed in webmaster tools.
The reason I say this is somewhat sloppy is because, you might find that you could have used regex in a lot of instances to better handle all those missing URLs.
But it may be a good way of staying on track with google, and handling the issues only as they arise so it does not feel like such a mammoth task.
-
Thanks Alan, yes they have good external links to many pages. They retail a very niche product and have a lot of forum, review, social type links. It might be though if need be they just have to focus mostly on 301s for the pages with those links. As best practise I am in favour of 301'ing regardless of external links as the link equity gets messed up and causes ranking issues, as in this case, as well as sending a signal to the engines about the amount of wasted resource they will use crawling a site with 1000s of 404s.
-
Thanks Donna & Luis. Luis is right i'm looking for a way for this not to be a mammoth manual task for their developer.
-
Thanks, the regex is a good idea and might be part of the solution for some urls at least but there seems to be some discrepancies in logic between old and new product urls and some of the new product urls are actually still the same as the old (which of course is fine).
-
Thanks Luis, unfortunately neither 1 or 2 are ideal.
1. I don't think there is much logic in the change of url structure between old and new product urls which makes that idea impossible.
2. Thats going to be a last resort
Andy
-
do you know if they had any external links?
If they don't have external links then I would just let them 404.
some people have some wired thoughts of what 301's do. They simply redirect a request, so a request o A is told to remake the request to to B, so the crawler will follow it that way and award the pagerank to the new page with a small loss on each request.If no external links what is there to gain? don't complicate your site with unnesasary redirects, there is a small argument that the pages may have been bookmarked at old url, but I think that argument is so weak I would not bother
-
Yeah. I heard him. I guess I'm saying "probably not".
I like how you're keeping us honest though Luis. I don't like it when people respond with what they want to say rather than with an answer to the specific question.
-
Donna,
Andy has been very specific about this: "WITHOUT it being a massive manual job" hehe
thanks for supporting my answer.
Luis
-
It really depends on the nature, link and traffic patterns of your site Andy. If the vast majority of those 2,000+ 404's are coming from pages that should never have been indexed in the first place, you can probably get away with Luis's 2nd suggestion. If they're differentiated, valuable, and show evidence of incoming links and traffic, you've got some work ahead of you.
You might be able to streamline the process by inventorying and grouping like pages, then doing group redirects. But I suggest you do some analysis first to determine whether the effort is warranted.
-
2000+ is a lot of URLs to work through. But you can most likely get through them quickly with a few good regular expression 301 redirects in your .htaccess
If you have a pretty consistent form from the old url to the new one, this will be a piece of cake.
ex:
old URL: this/was/coolnew URL: this/is/cool
However, if there is really no rhyme and reason to the newly formed URLs, this could end up taking a considerate amount of time.
I would look into writing 301 redirects with regular expressions in .htaccess (I'm assuming your server is and uses .htaccess)
There are a number of resources for doing this, and even one here at moz.com
https://moz.com/learn/seo/redirection -
Hello Andy,
1. Try this: http://webdesign.about.com/od/htaccess/ht/redirect-an-entire-site-using-htaccess.htm
2. Second/faster solution. You could add this line of code to your .htacess file (and all the current "404's users" will go to the homepage):
ErrorDocument 404 /
But pay attention... 404's are perfectly normal if the page no longer exists, for user experience you should only ever use a 301 redirect if the page that no longer exists is going to a equal page.. i.e about cars to cars, about rabbits to rabbits. Maybe the only solution is creating a 404 specific landing page for this (with links to different sections of your site)
Hope this helps,
Luis
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Sitemap and content question
This is our primary sitemap https://www.samhillbands.com/sitemaps/sitemap.xml We have a about 750 location based URL's that aren't currently linked anywhere on the site. https://www.samhillbands.com/sitemaps/locations.xml Google is indexing most of the URL because we submitted the locations sitemap directly for indexing. Thoughts on that? Should we just create a page that contains all of the location links and make it live on the site? Should we remove the locations sitemap from separate indexing...because of duplicate content? #
Intermediate & Advanced SEO | | brianvestSitemap Type Processed Issues Items Submitted Indexed --- --- --- --- --- --- --- --- --- 1 /sitemaps/locations.xml Sitemap May 10, 2016 - Web 771 648 2 /sitemaps/sitemap.xml Sitemap index May 8, 2016 - Web 862 730
0 -
Question about Syntax in Robots.txt
So if I want to block any URL from being indexed that contains a particular parameter what is the best way to put this in the robots.txt file? Currently I have-
Intermediate & Advanced SEO | | DRSearchEngOpt
Disallow: /attachment_id Where "attachment_id" is the parameter. Problem is I still see these URL's indexed and this has been in the robots now for over a month. I am wondering if I should just do Disallow: attachment_id or Disallow: attachment_id= but figured I would ask you guys first. Thanks!0 -
301 Redirect Showing Up as Thousands Of Backlinks?
Hi Everyone, I'm currently doing quite a large back link audit on my company's website and there's one thing that's bugging me. Our website used to be split into two domains for separate areas of the business but since we have merged them together into one domain and have 301 redirected the old domain the the main one. But now, both GWT and Majestic are telling me that I've got 12,000 backlinks from that domain? This domain didn't even have 12,000 pages when it was live and I only did specific 301 redirects (ie. for specific URL's and not an overall domain level 301 redirect) for about 50 of the URL's with all the rest being redirected to the homepage. Therefore I'm quite confused about why its showing up as so many backlinks - Old redirects I've done don't usually show as a backlink at all. UPDATE: I've got some more info on the specific back links. But now my question is - is having this many backlinks/redirects from a single domain going to be viewed negatively in Google's eyes? I'm currently doing a reconsideration request and would look to try and fix this issue if having so many backlinks from a single domain would be against Google's guidelines. Does anybody have any ideas? Probably somthing very obvious. Thanks! Sam
Intermediate & Advanced SEO | | Sandicliffe0 -
301 redirect subdirectory to new domain
I'm planning on using 301 redirects to spin out a subdirectory of my current website to be its own separate domain. For instance, I currently have a website www.website.com and my writers write tech news at www.website.com/news. Now I want to 301 redirect www.website.com/news to www.technews.com. Will this have any negative impact on SEO? What are some steps that I can take to minimize these impacts?
Intermediate & Advanced SEO | | Chris_Bishop1 -
Duplicate Content/ Indexing Question
I have a real estate Wordpress site that uses an IDX provider to add real estate listings to my site. A new page is created as a new property comes to market and then the page is deleted when the property is sold. I like the functionality of the service but it creates a significant amount of 404's and I'm also concerned about duplicate content because anyone else using the same service here in Las Vegas will have 1000's of the exact same property pages that I do. Any thoughts on this and is there a way that I can have the search engines only index the core 20 pages of my site and ignore future property pages? Your advice is greatly appreciated. See link for example http://www.mylvcondosales.com/mandarin-las-vegas/
Intermediate & Advanced SEO | | AnthonyLasVegas0 -
What Questions to Ask in SEO Interview
Tomorrow morning I have a call with an SEO company interested in doing some work with our company. Its a larger company who do a lot of SEO work, and seem to have good feedback around the place. But we have been very very white hat in our all our our SEO work so far, and some of their wording on their site talks about "Negotiations and acquisitions of link partners".. which gives me the feel they might be a little grey hat.. What are some good questions we should ask these guys to make sure what they are doing is legit, and not going to get us stung for anything? And what sort of work should we get them to do, if we are happy to take care of content creation, on page optimisation and social media activities? Thanks!
Intermediate & Advanced SEO | | timscullin0 -
Keyword Ranking Question
I have recently hired a SEO company to help with our keyword. My question is what are the best tools to use to verify what that are reporting. I can do an unpersonalized search, but I am likely still getting the my local results. I have been using the SEOmoz rank tracker in the past but for some reason it is not able to retrieve results over the past day or so. Are there any other good tools to check ranking for an exact url at the for non-localized, non personalized results? Thanks for the suggestions.
Intermediate & Advanced SEO | | fertilityhealth0 -
Filter after 301 and linked with high PR
Hi, I'd like to ask you what should I do in my situation. I've shorted my URLs from something like this: domain.com/module/action/type/id/keyword to this: domain.com/keyword After 301 SERP refreshed and position stayed the same (yea, lucky me :). After 2 days I got some hight PR links (4 and 5). After 8 days my new URL disapprear to one keyword. Now this take 6 days... I've removed these links and still no results. So the question is - what should I do? Remove new url and replace it with old one, get new links?
Intermediate & Advanced SEO | | sui0