Why might Google be crawling via old sitemap, when the new one has been submitted and verified?
-
We have recently relaunched Scoutzie.com and re-submitted our new sitemap to Google. When I look on Webmaster tools, our new sitemap has been submitted just fine, but at the same time, Google is finding a lot of 404s when crawling the site. My understanding, it is still using crawling the old links, which do not exists. How can I tell Google to refresh it's index and to stop looking at all the old links?
-
Yes it should. However, as Alan mentioned below, if you still have links pointing to the 404 pages, Google will always attempt to crawl them, and will keep you informed that you have errors.
If you do have external links to those 404 pages, you can 301 redirect them to an appropriate page using .htaccess. This way you'll keep the link value and also get rid of the Webmaster Tools error.
If you don't have any links to them, then yes, Google will eventually stop trying to crawl them.
-
It's very likely that we do. Given that I cannot track down a 1000+ links that now 404, will they eventually fall out by themselves, or do I have to tell Google that everything that's 404'ed should be dropped from crawl index? Thanks!
-
What if I simply pushed the new sitemap over the old one? In other words, scoutzie.com/sitemap is the same link, except now it contains the new map. That should be okay, right?
-
you may still have links pointing to those 404 pages on your site or externally. If not then eventually they will fall out of the index
-
Hey scoutzie,
This is actually covered pretty well in Joe Robison's blog post on fixing Webmaster Tools crawl errors: http://moz.com/blog/how-to-fix-crawl-errors-in-google-webmaster-tools
I'll quote the related info:
"One frustrating thing that Google does is it will continually crawl old sitemaps that you have since deleted to check that the sitemap and URLs are in fact dead. If you have an old sitemap that you have removed from Webmaster Tools, and you don’t want being crawled, make sure you let that sitemap 404 and that you are not redirecting the sitemap to your current sitemap."
Hope this helps, good luck!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Old links that need to be removed
We are having a problem trying to get rid of old links that have been removed from a website that no longer exists. See links below-There are several more that we need to deal with. We have tried putting them into Google Search Console, but they keep reappearing with Moz. Since our new website is hosted on Hubspot now and not on WordPress, which these links are referring to, we are trying to determine how to remove them from being indexed since they don't exist anymore? http://intouch-marketing.com/wp-content/uploads/2015/03/mas.png
Moz Pro | | InTouchMK
19 Redirect to 4xx http://intouch-marketing.com/wp-content/uploads/2014/03/TWITTER-HASHTAG.jpg
19 Redirect to 4xx http://intouch-marketing.com/wp-content/uploads/2016/04/Tweet-Activity-analytics-for-DCuevasIM-November-January.png
19 Redirect to 4xx0 -
PA up for one domain and still no movement on the other two!
Hey guys! Just wanted to see if anyone can help me on this one! I have zenory.co.nz the PA on this is 21 and DA is 10 however my other two domains com and com.au have been stagnant at both PA and DA at 1 - any reason why this might be?
Moz Pro | | edward-may0 -
All ranked pages on Googles SERP only links to home
I got a problem regarding my website called musik.dk, and hope you guys are able to help. I just got my first ranking results from Moz. My question is: All my keywords are linking to the home page, and not the artist page? For examples if I were to search on Rihanna on Google, then when musik.dk appears on the SERP, it only links to the home, and not musik.dk/rihanna.. This problem applies to all the given artists ranked on Googles SERP, it never shows the artist page itself, only links to home. Let me know if you need any information, and I gladly supply, this is kind of frustrating to me.. As a note: all pages besides the home has a PA of 1, and it doesn't really seem to change. UT3OBHr kVp7z38
Moz Pro | | Morten_Hjort0 -
I'm New To Moz What To Focus On First
Hello, Recently signed up to MOZ for the sites we operate in the UK. I wondered what folks would recommend I focus on first when starting to use MOZ for the first time for sites SEO? Cheers Stuart
Moz Pro | | Urban331 -
Help: keyword position on google.ro (country: Romania)
Hello there, I have a stupid question: Why i cannot grow up, or google not do anything, when all indicators are better than a competitor. If you will see this link: http://analytics.moz.com/search/analyze-keyword-report/542983.308651/huse samsung galaxy s4 , you will see that, they are 1st on that keyword for better than 2 month. And we have better PA / Links / DA. Thank you in advance, Shanaki
Moz Pro | | Shanaki0 -
How to fix the Crawl Diagnostics error and warnings
hi im new to the seo world and i dont know a lot about it , so after my site get crawled i found 1 error and 151 warning and 96 notices , it that bad ?? and plz cam someone explain to me how to fix thos problem , a will be very thankful
Moz Pro | | medlife0 -
New Backlinks by Week?
One of my competitors has recently got a surge of links and I need to work out where they've been focusing their efforts to make sure I don't get left behind. The only method I've really got at the moment is looking at saved reports from OSE and trying to find new links between them. This is obviously time consuming and has some odd results (Old content showing up only on newer reports for example and have to ensure that each weeks report is generated exactly the same). What I really need is a weekly output of where 2 or 3 websites have been gaining links. Can anyone recommend any tools that can help me with this? Thanks
Moz Pro | | Stinkyink0 -
Google rankings dropped for some keywords
http://www.funktiongolf.co.uk uk google Our site ranked well for these words funky golf trendy golf clothes Now these keywords do no rank at all for my website. I have not been penalised by google as i emailed them. The only thing I did was add best seller products to my main page. Now my about us page ranks far higher than my index page Please Help
Moz Pro | | funktiongolf0