Seek help correcting large number of 404 errors generated, 95% traffic halt
-
Hi, The following GWT screen tells a bit of the story:
site: http://bit.ly/mrgdD0
http://www.diigo.com/item/image/1dbpl/wrbp
On about Feb 8 I decided to fix a large number of 'duplicate title' warnings being reported in GWT "HTML Suggestions" -- these were for URLs which differed only in parameter case, and which had Canonical tags, but were still reported as dups in GWT.
My traffic had been steady at about 1000 clicks/day.
At midnight on 2/10, google traffic completely halted, down to 11 clicks/day.
I submitted a recon request and was told 'no manual penalty'
Also, the 'sitemap' indexes in GWT showed 'pending' for 24x7 starting then.
By about the 18th, the 'duplicate titles' count dropped to about 600 or so... the next day traffic hopped right back to about 800 clicks/day - for a week - then stopped again, down to 10/day, a week later, on the 26th.
I then noticed that GWT was reporting 20K page-not found errors - this has now grown to 35K such errors!
I realized that bogus internal links were being generated as I failed to disable the PHP warning messages.... so I disabled PHP warnings and fixed what I thought was the source of the errors.
However, the not-found count continues to climb -- and I don't know where these bad internal links are coming from, because the GWT report lists these link sources as 'unavailable'.
I'v been through a similar problem last year and it took months (4) for google to digest all the bogus pages ad recover. If I have to wait that long again I will lose much $$.
Assuming that the large number of 404 internal errors is the reason for the sudden shutoff...
How can I a) verify the source of these internal links, given that google says the source pages are 'unavailable'..
Most critically, how can I do a 'RESET" and have google re-spider my site -- or block the signature of these URLs in order to get rid of these errors ASAP??
thanks
-
Hello Rand, I've been facing a similar problem with my site. I'd really appreciate your response here - http://www.seomoz.org/q/help-fixing-the-traffic-drop-that-started-on-4-september-2012.
-
I wouldn't feel too confident that the numbers and dates Google's showing you are precise or accurate. In fact, we've seen times when GWMT is considerably off. I'd watch how Google crawls your site and look at search traffic to your pages - those are likely leading indicators that things are/will be fixed.
-
Thanks for the replies guys - - I had run Xenu on the site and it found no broken links... but still GWT error count continues to climb, and as of today
Google released a MUCH improved timeline view for the error count --- problem is, it's still showing 58K errors as of yesterday and climbing, long after I fixed them - and it wont show me where it thinks the source is...
These errors are all on internal pages BTW..
Heres the new google view
http://awesomescreenshot.com/0ef1gy6c7
The new GUI also includes a way to mark errors 'fixed' -- one by one!! I need to mark 60 thousand at once!
Also I can see the date these errors started appearing and it just doesnt make sense given that is the day my traffic started reappearing as well..
-
I agree with Rand's suggestions. I just ran a Screaming Frog crawl of the whole site on 10,233 links, 8997 URLs and got no 404s. So I think it's pretty safe to assume you've fixed the 404 issue. Here's the output of the crawl in case you'd like it for a reference: http://www.sendspace.com/file/7zui0v
I'd say:
- Definitely clean up and resubmit your XML sitemap
- Double check your backlink profile with Open Site Explorer and MajesticSEO to be sure that there aren't sites linking to URLs that no longer exist. If you find any of these make sure to 301 redirect them. Just take all the target URLs and dump them into Screaming Frog in list mode. All the links from OSE point to your homepage so they are not an issue, I don't have access to Majestic right now so I couldn't run those for you.
- You can now Submit pages in Google Webmaster Tools as well in the Fetch as Googlebot section. So you may consider submitting some of the new pages the site generates in addition to your reconsideration request to help get Google to re-crawl and find the 404s are gone.
Good luck man and please let us know if nothing changes after you implement these fixes.
-Mike
-
Hi Mark - wow, sounds really rough. I've got a few suggestions:
- First off, you need to make 100% sure that you've actually fixed the issue and that the internal links are pointing to the right places AND any old URLs that may have had internal/external links are either rel=canonicaling or 301 redirecting to the correct, updated locations.
- You might try using a few tools to verify this, including the SEOmoz Crawl Test http://pro.seomoz.org/tools/crawl-test and Screaming Frog: http://www.screamingfrog.co.uk/seo-spider/
- When you are ready, submit new XML Sitemaps to Google with the proper URLs. Make sure you've deleted/removed your old ones.
- You can also send the reconsideration request again, indicating that while you're aware this isn't a penalty, you have realized some technical/navigation issues on the site and believe you've now fixed these.
Hope this helps and wish you the best of luck!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How do i fix fatal error message?
I Am Trying To Remove A Robots.txt code i put in my root domain a while back because i didn't know what i was doing. everytime i enter my domain (domain.com/robots.txt) i get a fatal error message. How do I fix this fatal error message? ipALV2z
Technical SEO | | icebergsal0 -
Does 404 hurts my website.?
Does 404 hurts my website.? My website reporting 404 crawl errors on pages that have never been part of the website. Site showing lot of 404 errors and all errors from Blog.We didn't create any pages .Automatically pages are created and showing 404 (Not Found) error response..Please any one help me to identify those issues.?
Technical SEO | | nlogix0 -
404 not found page appears as 200 success in Google Fetch. What to do to correct?
We have received messages in Google webmaster tools that there is an increase in soft 404 errors. When we check the URLs they send to the 404 not found page:
Technical SEO | | Madlena
For example, http://www.geographics.com/images/01904_S.jpg
redirects to http://www.geographics.com/404.shtml.
When we used fetch as Google, here is what we got: .
#1 Server Response: http://www.geographics.com/404.shtml
HTTP/1.1 200 OK Date: Thu, 26 Sep 2013 14:26:59 GMT
What is wrong and what shall we do? The soft 404 errors are mainly for images that no longer exist in the server. Thanks!0 -
Site getting referral traffic from its self
Buon giorno from zero degrees C freezing fog Wetherby UK On this site http://www.collegeofphlebology.com i ran a referral report via Google Analytics and was surprised ro see referral traffic being counted from its own url, illustration here http://i216.photobucket.com/albums/cc53/zymurgy_bucket/referral-anomoly.jpg So my question is please how can a site get referral traffic from the same url? Grazie Tanto, David
Technical SEO | | Nightwing0 -
Best XML Sitemap generator
Do you guys have any suggestions on a good XML Sitemaps generator? hopefully free, but if it's good i'd consider paying I am using a MAC so would prefer a online or mac version
Technical SEO | | kevin48030 -
Please help....
Hi Guys! Ok a bit of a funny one here which is causing a confusion between us and a web designer and I was wondering if anyone on here might be able to help. Just a bit of back ground for you, the website has been built on Concrete 5 and when we tried to building a sitemap we found over 110,000 pages. When we spoke to the web designer they have told us that within Google webmaster tools, Google has only indexed 58. But.... (and this is where things get a little confusing, so bare with me.) I thought that cant be right so into the Google search bar I put in site:www.sitename.co.uk and had 217 results appear. So google cant have just 58 pages indexed, right? So after speaking to the designer he then posted on the Concrete 5 help forum, to try and help figure it out. I have posted his exact forum post below that the web designer has asked: I'm having some issues where a site we are working on seems to be making multiple pages going to the same page. An SEO specialist has run a report and found a number of duplicate pages created by C5. We are concerned that this is going to dilute or worse penalise the way google sees the site. http://www.sitename.co.uk/
Technical SEO | | NoisyLittleMonkey
[http://www.sitename.co.uk/index.php?cID=?akID[155]atSelectOptionID...
[http://www.sitename.co.uk/index.php?cID=?akID[155]atSelectOptionID...
[http://www.sitename.co.uk/index.php?cID=?akID[155]atSelectOptionID... Is there a way of stopping google from accessing these duplicate 'cID' pages and stop them being made? Also is there a way of getting rid of the ones that are there? We've done a number of sites in C5 and are beginning to get concerned about this... So I guess my question is: If I can access the same content via 4-5 different cID's is that classed as duplicate content? Thanks in advance guys, and any help would greatly appreciated. 🙂0 -
Crawl Errors In Webmaster Tools
Hi Guys, Searched the web in an answer to the importance of crawl errors in Webmaster tools but keep coming up with different answers. I have been working on a clients site for the last two months and (just completed one months of link bulding), however seems I have inherited issues I wasn't aware of from the previous guy that did the site. The site is currently at page 6 for the keyphrase 'boiler spares' with a keyword rich domain and a good onpage plan. Over the last couple of weeks he has been as high as page 4, only to be pushed back to page 8 and now settled at page 6. The only issue I can seem to find with the site in webmaster tools is crawl errors here are the stats:- In sitemaps : 123 Not Found : 2,079 Restricted by robots.txt 1 Unreachable: 2 I have read that ecommerce sites can often give off false negatives in terms of crawl errors from Google, however, these not found crawl errors are being linked from pages within the site. How have others solved the issue of crawl errors on ecommerce sites? could this be the reason for the bouncing round in the rankings or is it just a competitive niche and I need to be patient? Kind Regards Neil
Technical SEO | | optimiz10 -
I need some HTACCESS help (Magento)
Hi Guys, I need some help on this htaccess issue in Magneto. So here is what I am trying to do: I wanted to change mysite.com/index.php/etc to mysite.com/etc so I turned on the web friend URLS. That did that, BUT there are still two versions of every page on the site. www.mysite.com/etc and mysite.com/index.php/etc So that isn't good for SEO. So then I applied a 301 matching redirect, RedirectMatch 301 /index.php/(.*) http://www.mysite.com/$1 That solved that problem. But now I am not able to log into the admin. It is mysite.com/index.php/pg45admin. It should redirect to mysite.com/pg45admin but the page just hangs.... It goes into a continuous loop. I tired using the custom URL and then the site crashed and I had to redo it. So what do I need to do for this to work?
Technical SEO | | netviper0