Webmaster tools crawl errors
-
Hi there,
iv been tracking my webmaster tools crawl errors for a while now(6 months) and im noticing some pages that are far gone 404 are still poping out on the crawl errors. - that pages have no data for xml linking, and remote linking are from pages that are far gone 404 also.
that pages have 404 error page + redirect to homepage, and google still notice them with old cache content.
does someone have a clue why is this happening?
-
Thank you very much Dana for the superb answer!
any clue for how much this errors are critical for my website seo? (is this problem worth fixing)
-
Hi!
Have you verified that there is a proper 301 redirect from the old URL's? If, for example, there is a 302 instead, I wouldn't be surprised if Google kept the old info in the index, as you have sort of said "I'll be back soon at this old address, just wait a minute (or year)".
Do you have an example URL for us that we could take a look at?
-
Hi Dana,
If your looking for someone to confirm what Dana said she is 100% right.
If your using a data base like WordPress it is posable for it to hold old pages and serve them. Make sure your host knows of your problem if you are using a CMS.
Hope this helps,
Thomas
-
Yes, I understand this issue very well and have seen it many times. Most often, it is happening because another of your pages that is being indexed is still referencing those pages. Please forgive Google, it is but a humble, not-so-intelligent bot (despite what the world would have you think). If you keep referencing a page that doesn't exist, Googlebot will say to itself "but it does exist! it does!" and keep indexing it, despite the 404 error. I mean, for all Google knows, it is your intention to bring that page back and maybe you just screwed up and it is 404-ing, you know?
Here's the remedy:
Do a content audit. Here's a great post on how to do that: http://www.distilled.net/blog/seo/how-to-perform-a-content-audit/
You will discover many things, without a doubt, including pages that are linking to these 404 pages. Decide what to do with those pages, i.e. nix them, re-build them, whatever. If you have pages in Google that you really do want out of the index and those same pages are 404-ing....simply go to your Google Webmaster Tools Account and in the left nav select "Google Index" and then "Remove URLs." Then simply click the box that says "create a new removal request" and enter the desired URL into that Box. Provided your page meets the requirements (and if it's producing a 404 error it does) Google will prioritize its removal. Under no circumstances should this be confused with the disavow tool. Just to make that clear.
I hope this helps. Any questions and I'm happy to help further if I can.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Data missing in the Attribution Model Comparison Tool
Hi, I analysed our conversion data in the model comparison tool. However, there seem to be o lot of data missing - overall conversion value and conversion value for every channel is much lower than shown in the regular acquisition report. Does anybody else experience the same problem? Thanks. Veronika
Reporting & Analytics | | healthpostnz0 -
Webmaster Tools Records
Hey, I'm trying to find a way of getting data from Googles Webmaster Tools and recording it in either a database or spread sheet. Is there any software out there that I can use to do this or would i have to look at developing it my self? Thanks, Luke.
Reporting & Analytics | | NoisyLittleMonkey0 -
What are some good SERP visibility tools?
With keyword data from Google on its way out, we're losing our ability to discover what keywords a website is ranking for (you can do ranking checks, but only if you already know what keywords the site is ranking for). What are some good SERP visibility tools? (i.e. you enter the domain name, and the tool lists which keywords the site ranks for organically, etc.) I know SpyFu offers this, but they seem to be PPC focused and perhaps not very accurate. I know other companies offer it, but I'm blanking on which ones.
Reporting & Analytics | | AdamThompson0 -
My GWT tells me that verification has failed numerous occasions - will this stop my site being crawled?
I launched www.over50choices.co.uk 6 weeks ago and have had trouble with google indexing and crawling all pages. It tells me 143 submitted & 129 Indexed, but the site has 166 pages? It still shows the old home page image in GWT - which is v annoying! Whilst the site is verified by GA & HTML Tag, it tells me in the Verification section that "reverification failed" on numerous occasions - they seem correspond with when google trys to process the site map. Is this a coincidence ie verification fails when its trying to process the site map, which in turn is leaving me with an out of date site map and therefore not all my pages submitted or crawled? Or will this not effect the googles ability to crawl the site? Your help please. Ash
Reporting & Analytics | | AshShep10 -
Google Webmaster Tools - spike in 'not selected' under Index Status
Hi fellow mozzers Has anyone seen a huge shift in the number of pages 'Not Selected' under Index Status in Google WMT, and been able to identify what the problem has been? My new client recently moved their site to wordpress - and in doing so the number of pages 'not selected' rose from ~200 to ~1100, It was high before but is ridiculous now. I am thinking there must be a new duplicate content issue which should be cleaned up in my quest to improve their SEO. Could it be the good old WP tag/category issue? In which case I won't worry as Joost is doing its job of keeping stuff out of the index. There are loads of image pages which could well appear as dupe as have no content on them (i do need to fix this), but Google is already indexing these so doesn't explain the ones 'not selected'. I've tried checking dupe title tags but there are very few of them so that doesn't help Any other ideas of how to identify what these problem pages maybe? Thanks very much! Wendy
Reporting & Analytics | | Chammy0 -
Rank tracker tool - seomoz
Hi All, I have noticed that ranks in the tool -rank tracker are not correct for my keywords sometimes. Does anybody else has experienced this? Isn't it real time updated? What other free rank tracker tool you guys would suggest which can send you weekly accurate updates? Thanks
Reporting & Analytics | | EG0CENTRIX0 -
Is Google Webmaster Tools Index Status Completely Correct?
So I was thrilled when Webmaster tools released the little graph telling you how many pages are indexed. However, I noticed with one of my sites, that it was actually going down over the past year. I've only been doing the SEO on the site for a few months, so it wasn't anything I did. The chart is attached to this post. Now here's the funny part. I haven't really noticed anything out of the ordinary for keyword ranking dropping off. I also tested my most recent page that I put up. 3 days after I posted it, I could find it in a Google search. Shouldn't this mean that's it's indexed? I can also find any other page I've posted in the last few months. Another oddity is that I submitted a sitemap a while ago when the site was only 22 pages. The sitemap index count says 20 of those pages are indexed. The chart only says that there are 3 indexed pages right now. However, I can clearly find dozens of pages in Google searches. Is there something I'm missing? Is my chart for this website broken? Should I report this to Google? Has anyone had a similar issue? decreaseIndex.png
Reporting & Analytics | | mytouchoftech0 -
Increase number of pages crawled
Only one page is being crawled, how do I increase the number to include most of our site?
Reporting & Analytics | | NorthCoast0