Crawl errors for pages that no longer exist
-
Hey folks,
I've been working on a site recently where I took a bunch of old, outdated pages down. In the Google Search Console "Crawl Errors" section, I've started seeing a bunch of "Not Found" errors for those pages. That makes perfect sense.
The thing that I'm confused about is that the "Linked From" list only shows a sitemap that I ALSO took down. Alternatively, some of them list other old, removed pages in the "Linked From" list.
Is there a reason that Google is trying to inform me that pages/sitemaps that don't exist are somehow still linking to other pages that don't exist? And is this ultimately something I should be concerned about?
Thanks!
-
Thanks for the question, this can definitely be annoying for webmasters!
Unfortunately, bots can don't everything in parallel. They have to take steps...
Step 1. Take List #1 of links.
Step 2. Crawl those links and build List #2.
Step 3. Crawl List #3 and build List #4...Now, sometimes it doesn't follow that same order. Let's say that in Step 3 it finds a bunch of pages with unique content. Maybe the next time around, it goes and checks some of those links in Step 3 without first checking if they were still linked. Why start the crawl all the way from the beginning again when you have a big list of URLs?
But, this creates a problem. When some of those links it crawled in Step 3 aren't there any more, Google will tell you they aren't there and tell you how they originally found them (which happened to be from a page in List #1). But what if Google hasn't checked that link in List #1 recently? What if you just removed it too?
Well, for a little while, at least, you will end up with errors.
Now, here comes the real rub - how long will it take for Google to find and correct that message it left you in the crawl report? Days? Weeks? Months? Who knows. Your best bet is to mark them as fixed and force Google to keep rechecking. Eventually, they will figure it out.
TL;DR; it is a data freshness and reporting issue that isn't your fault and isn't worth your time.
-
No - Google is just showing how slow it is when updating data in Webmaster tools.
Don't worry - if you wait long enough they'll go away. You could also mark them as solved (do this only if you are sure that there are no links pointing to these pages - to check if your internal linking is ok Screaming Frog is great tool)
Dirk
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Which page speed tools do you trust?
We are at a loss. We have been following Google Pagespeed Insights which gives us horrible marks. Nothing we do seems to really make a difference. GTMetrix and Pingdom give us much better scores. We don't know what to do...
Reporting & Analytics | | HashtagHustler0 -
How does Google measure page position in Webmasters?
Does anyone know exactly how Google measures page position in Webmaster Tools? For example: In Google Webmaster Tools, we had a product which on the 22/12/15 was at position 7, and then dropped to position 112 on the 30/12/15. It then rose back up to position 7 on the 6/01/16 and then down to position 25 on the 16/01/16. What does this mean and why?
Reporting & Analytics | | CostumeD0 -
No-indexed pages are still showing up as landing pages in Google Analytics
Hello, My website is a local job board. I de-indexed all of the job listing pages on my site (anything that starts with http://www.localwisejobs.com/job/). When I search site:localwisejobs.com/job/, nothing shows up. So I think that means the pages are not being indexed. When I look in Google Analytics at Acquisition > Search Engine Optimization > Landing Pages, none of the job listing pages show up. But when I look at Acquisition > Channels > Organic and then click Landing Page as the primary dimension, the /job pages show up in there. Why am I seeing this discrepency in Organic Landing pages? And why would the /job pages be showing up as landing pages even though they aren't indexed?
Reporting & Analytics | | mztobias0 -
High Temporary Redirects: Login required pages
Noticed something interesting, a high temporary redirect report from Moz. Reviewing the pages they are caused by the user having to login and getting redirected. I can see the returnto query in the URL too. My thoughts: Since a login is required and the user is being redirected, these should remain 302 and not 301. I tested my Google Analytics account to **Exclude URL Query Parameter **returnto, just to see if it affected traffic. It didn't, I mean I don't see urls duplicated with the parameter anymore, just grouped together, so traffic is still being counted. I'm going to wait 1 more day and see what impact the GA traffic is before applying the exclusion to my true Google Analytics profile. This got me thinking, I should probably exclude this parameter from Google and Bing Webmaster Tools, that way Google/bing won't read those urls. Now does Moz's crawler follow that? Do you think that would change my moz crawl diagnostic report because I told Google/Bing crawlers to exclude that parameter. What do you think of my approach to reduce these high temporary redirects reported by Moz? Will it work? Has it plagued you?
Reporting & Analytics | | Bio-RadAbs0 -
How to hook up a ppc campaign to a google + Page
Greetings,
Reporting & Analytics | | Nightwing
Sometimes you just want to give Google a big slap for making straight forward requests damn impossible. So all i ma trying to ad is point a ppc ad at this Google + account <a>https://plus.google.com/118393512656496298734#118393512656496298734/posts</a> But i get a warning sign saying:
"The URL must be for a Google+ page, not a personal profile" I then spend half an hour tring to find a Google + page but get no where fast 😞 Warning message illustrated here:
http://i216.photobucket.com/albums/cc53/zymurgy_bucket/google-page-plus_zps46ff995a.jpg So my question is please how to a get the Google + page for this account:
<a>https://plus.google.com/118393512656496298734#118393512656496298734/posts</a> Any insights welcome!
David0 -
Confirmation page gets hit multiple times by some users. How I can I segment out unique visits?
Hi All, I'm web marketing manager at http://www.evenues.com which is like an AirBnB for meeting space. When calculating the number of bookings for our meeting spaces, I've set up a goal in analytics with the confirmation page as the goal URL. The problem is, it seems that some users are looking at the same confirmation page several times. We have unique URLs for each confirmation page, but some users seem to be visiting these unique pages more than 2 to 5 times. This skews our numbers a bit. This makes things a bit problematic when it comes to segmenting visitors. is there anything we can so that each unique URL visited only counts once? Thanks, Kenji
Reporting & Analytics | | eVenuesSEO0 -
Setting up Webmaster Tools correctly - naked domain DNS error and sub-domains question
I'm trying to get our domain (verdantly.com) set up correctly in Google Webmaster Tools. Currently, I have three "sites" setup: blog.verdantly.com (wordpress.com blog redirected to this subdomain) www.verdantly.com verdantly.com The subdomain blog and www show up without errors. However, the naked domain shows a DNS error. I've checked the DNS settings at the registrar and don't see any issues. So here are my questions: 1. Am I correct in setting up the naked domain AND the subdomains separately in Webmaster tools? 2. How do I track down / resolve the source of the DNS errors at the naked domain? Thanks!
Reporting & Analytics | | letsdothis0 -
Google Analytics - In-Page Analytics
I had a strange thought waking up this morning, and was curious to hear other people's opinions on it. In Google Analytics, under Content > In-Page Analytics, Google shows what links on your site pages get clicked and how many times plus other metrics. Do you think they use that data for ranking back links so-to-speak? What I mean is, say I had a back link to my site on example.com, and example.com had google analytics installed. Google can see through google analytics whether my link has been clicked on. Say that my link gets no clicks, do you think that Google would use that metric against my site deeming it "not popular" or "not a good resource", even if example.com was a very popular site? And it could work the other way. Say my link got thousands of clicks on example.com, do you think that Google might use that to promote my site? I couldn't find any other discussion on this anywhere, so am not sure if people have already thought about this.
Reporting & Analytics | | THB0