How to Destroy Old 404 Pages

[TheOceanAgency]

Hello Mozzers,

So I just purchased a new domain and to my surprise it has a domain authority of 13 right out of the box (what luck!).  I needed to investigate.  To make a long story short the domain used to be home to a music blog that had hundreds of pages which of course are all missing now.  I have about 400 pages on my hands that are resulting in a 404.  How or what is the best method for eliminating these pages.

Does deleting the Crawl Errors in Google Webmaster Tools do anything?

Thanks

[TheOceanAgency]

What a thorough response!  I'm in the Option B scenario.  The old content has nothing to do with my site so I don't need to redirect the old URLs.  I will just wait out Google crawling those 404s.

Thanks!

[CleverPhD]

You have a few options here.  Option A is if you are going to build a site that will have similar topic based content as the old one and you want to use a larger portion of that domain authority from the old site to the new.

1. Pull those 404 errors from GWT in a spreadsheet.   This gives you a corpus of links to work with.

2. Go into Bing WT and they have a way to browse what they have and had indexed.  What is nice here is that Bing will tell you what URLs (even old 404s) have links to them.

3. Run your links through Open Site Explorer. You can then also get linking data, FB and Twitter data in addition to OSE data on the old URLs

4. If need be, run the more important dead URLs through the Wayback Machine http://archive.org/web/web.php  you can now even see what the actual content was on the old URLs.

5. After doing all of this, pretty quick you should be able to see if there were any authority pages on the site that have now expired and you also know what those pages were about via the wayback machine.

6. On the authority pages, create new pages on the new site that have to do with the same topic, i.e. semantically related to the old page.

7. 301 the old authority pages to the new authority pages.

8. The rest of the URLs you can just let them 404.  They will continue to 404 several time until Google drops them.  I would leave them in GWT as over time they should drop out as Google starts to ignore those pages, this may take a few months.  You can then just check GWT for any new 404s that might show up from the new site and you need to deal with.

One thing to note on all of this.  You may have to let the old sitemap 404 vs redirecting the sitemap.

http://moz.com/blog/how-to-fix-crawl-errors-in-google-webmaster-tools

"One frustrating thing that Google does is it will continually crawl old sitemaps that you have since deleted to check that the sitemap and URLs are in fact dead. If you have an old sitemap that you have removed from Webmaster Tools, and you don’t want being crawled, make sure you let that sitemap 404 and that you are not redirecting the sitemap to your current sitemap."

If you delete the 404s from GWT the next time Google spiders the old pages they will just show up again, up to you then.

Option B - if you dont care about the old pages, just let them 404 as mentioned above, but be aware of the issue with old sitemaps.  You can check the Google index for old URLs in the SERPs or also if you look into GWT and look for data on your Search Traffic.  Make sure that the old URLs are not showing up under your Search Queries.