How to Destroy Old 404 Pages
-
Hello Mozzers,
So I just purchased a new domain and to my surprise it has a domain authority of 13 right out of the box (what luck!). I needed to investigate. To make a long story short the domain used to be home to a music blog that had hundreds of pages which of course are all missing now. I have about 400 pages on my hands that are resulting in a 404. How or what is the best method for eliminating these pages.
Does deleting the Crawl Errors in Google Webmaster Tools do anything?
Thanks
-
What a thorough response! I'm in the Option B scenario. The old content has nothing to do with my site so I don't need to redirect the old URLs. I will just wait out Google crawling those 404s.
Thanks!
-
You have a few options here. Option A is if you are going to build a site that will have similar topic based content as the old one and you want to use a larger portion of that domain authority from the old site to the new.
-
Pull those 404 errors from GWT in a spreadsheet. This gives you a corpus of links to work with.
-
Go into Bing WT and they have a way to browse what they have and had indexed. What is nice here is that Bing will tell you what URLs (even old 404s) have links to them.
-
Run your links through Open Site Explorer. You can then also get linking data, FB and Twitter data in addition to OSE data on the old URLs
-
If need be, run the more important dead URLs through the Wayback Machine http://archive.org/web/web.php you can now even see what the actual content was on the old URLs.
-
After doing all of this, pretty quick you should be able to see if there were any authority pages on the site that have now expired and you also know what those pages were about via the wayback machine.
-
On the authority pages, create new pages on the new site that have to do with the same topic, i.e. semantically related to the old page.
-
301 the old authority pages to the new authority pages.
-
The rest of the URLs you can just let them 404. They will continue to 404 several time until Google drops them. I would leave them in GWT as over time they should drop out as Google starts to ignore those pages, this may take a few months. You can then just check GWT for any new 404s that might show up from the new site and you need to deal with.
One thing to note on all of this. You may have to let the old sitemap 404 vs redirecting the sitemap.
http://moz.com/blog/how-to-fix-crawl-errors-in-google-webmaster-tools
"One frustrating thing that Google does is it will continually crawl old sitemaps that you have since deleted to check that the sitemap and URLs are in fact dead. If you have an old sitemap that you have removed from Webmaster Tools, and you don’t want being crawled, make sure you let that sitemap 404 and that you are not redirecting the sitemap to your current sitemap."
If you delete the 404s from GWT the next time Google spiders the old pages they will just show up again, up to you then.
Option B - if you dont care about the old pages, just let them 404 as mentioned above, but be aware of the issue with old sitemaps. You can check the Google index for old URLs in the SERPs or also if you look into GWT and look for data on your Search Traffic. Make sure that the old URLs are not showing up under your Search Queries.
-
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
I'm looking for a bulk way to take off from the Google search results over 600 old and inexisting pages?
When I search on Google site:alexanders.co.nz still showing over 900 results. There are over 600 inexisting pages and the 404/410 errrors aren't not working. The only way that I can think to do that is doing manually on search console using the "Removing URLs" tool but is going to take ages. Any idea how I can take down all those zombie pages from the search results?
Intermediate & Advanced SEO | | Alexanders1 -
May integrating my main category page in the index page improve my ranking of main category keyword?
90% of our sales are made with products in one of our product categories.
Intermediate & Advanced SEO | | lcourse
A search for main category keyword returns our root domain index page in google, not the category page.
I was wondering whether integrating the complete main category directly in the index page of the root domain and this way including much more relevant content for this main category keyword may have a positive impact on our google ranking for the main category keyword. Any thoughts?1 -
How does Googlebot evaluate performance/page speed on Isomorphic/Single Page Applications?
I'm curious how Google evaluates pagespeed for SPAs. Initial payloads are inherently large (resulting in 5+ second load times), but subsequent requests are lightning fast, as these requests are handled by JS fetching data from the backend. Does Google evaluate pages on a URL-by-URL basis, looking at the initial payload (and "slow"-ish load time) for each? Or do they load the initial JS+HTML and then continue to crawl from there? Another way of putting it: is Googlebot essentially "refreshing" for each page and therefore associating each URL with a higher load time? Or will pages that are crawled after the initial payload benefit from the speedier load time? Any insight (or speculation) would be much appreciated.
Intermediate & Advanced SEO | | mothner1 -
Why has my home page replaced my sub-category page for set of keywords? Happened 2x in last 2 weeks for day or so only to fix itself. What is going on?
Today I noticed a really weird problem. Our LED Step Lights page (https://www.pegasuslighting.com/led-step-lights.html) has been replaced in the search results with our home page. See screenshot below. As I started to research what was going on, I noticed that this same thing must have happened on January 26 and 27 because in my Analytics I can see that our LED Step Lights sub-cat page had a sudden drop in traffic on those two days only to bounce back again on the 28th. See screenshot below. Our LED Step Lights page has had no changes in content, meta information, or anything in months. We have done no recent link building to this page in years. I don't understand what is going on. This is a popular page for us generating decent traffic. I really don't understand what is going on or even how to try and resolve this problem. I checked our Search Console. No messages. No manual web spam actions. Nothing to suggest that anything is going on except for the weird drops in traffic. Has anyone ever seen this happen before? Does anyone have any ideas as to what may be going on? serp-led-step-lights.png organic-traffic-drops.png search-console-led-step-lights.png
Intermediate & Advanced SEO | | cajohnson0 -
Sitemaps and dynamic pages
Hi all, I have a gigantic website and they are adding another subdirectory to it. My question is regarding html sitemaps for better optimisation. 1. Should a keyword focussed front end (html) sitemap be made for all the dynamic URLs or 2. Should a category focussed front end (html) sitemap be made for all the dynamic URLs what would be your approach to doing a sitemap with thousands of pages with a structure like Directory > Sub directory > Subdirectory > Files
Intermediate & Advanced SEO | | Malika10 -
Should I merge these pages
I have this business and am not sure if I should have a separate page for all of the different roofing subservices or if i should put them all on one page. Even though they are separate, but related services, I feel they could end up competing against one another If I merge them I will also have more related and keyword rich content on one page that I could focus my efforts on.
Intermediate & Advanced SEO | | Atomicx0 -
Do search engines crawl links on 404 pages?
I'm currently in the process of redesigning my site's 404 page. I know there's all sorts of best practices from UX standpoint but what about search engines? Since these pages are roadblocks in the crawl process, I was wondering if there's a way to help the search engine continue its crawl. Does putting links to "recent posts" or something along those lines allow the bot to continue on its way or does the crawl stop at that point because the 404 HTTP status code is thrown in the header response?
Intermediate & Advanced SEO | | brad-causes0 -
On Page vs Off Page - Which Has a Greater Effect on Rankings?
Hi Mozzers, My site will be migrating to a new domain soon, and I am not sure how to spend my time. Should I be optimizing our content for keywords, improving internal linking, and writing new content - or should I be doing link building for our current domain (or the new one)? Is there a certain ratio that determines rankings which can help me prioritize these to-dos?, such as 70:30 in favor of link-building? Thanks for any help you can offer!
Intermediate & Advanced SEO | | Travis-W0