Google Webmaster tools: Sitemap.xml not processed everyday
-
Hi,
We have multiple sites under our google webmaster tools account with each having a sitemap.xml submitted
Each site's sitemap.xml status ( attached below ) shows it is processed everyday except for one
_Sitemap: /sitemap.xml__This Sitemap was submitted Jan 10, 2012, and processed Oct 14, 2013._But except for one site ( coed.com ) for which the sitemap.xml was processed only on the day it is submitted and we have to manually resubmit every day to get it processed.Any idea on why it might?thank you
-
My initial reaction was that this is more likely technical than something Google is doing - checking the load-time is a good idea. Make sure the sitemap validates and there's nothing odd about it. If you manually re-submit it, does it seem to take?
-
Just so I am clear, you have been waiting and finally Google processed it, or was it sitting there and someone took an action which caused Google to process it?
I am surprised that nothing happened for nearly two years. Has the site had traffic, etc.? Any warnings, manual actions, etc.?
Thanks
-
Thanks Paul,
On custom crawl setting, I just verified and they remain same same across all our sites.
Yes, sitemap is dynamic but rendered via cache which refreshed when new content get published, will check on the load time.
thank you
-
Thanks Robert,
All our sites are getting indexed for sure, but one site ( coed.com ) sitemap.xml on GWT says it was processed only on the day it submitted, while other sites ( collegecandy.com and bustedcoverage.com ) sitemap.xml GWT was getting processed everyday
Our sitemap.xml on all sites updates automatically when new content get published so I believe it has to be processed every day
-
Any chance there's been a custom crawl setting accidentally added to your Google Webmaster Tools, Robert? Some devs do this during development or it can happen accidentally.
Also, your sitmemap takes a ridiculously long time to load - well over 15 seconds for me and over 18 seconds using webpagetest.org. It could be that Google simply isn't waiting for the page to load when it tries to visit. If the sitemap's being generated dynamically, you may have a rendering problem. Otherwise there's something borked when a 50kb file takes that long.
Might also want to try submitting it through Bing Webmaster Tools and see if they are better able to index it consistently for comparison?
Bit of a head-scratcher. Hope that gives you a starting point.
Paul
-
Robert,
There is no screenshot attached, but I am unaware of sitemaps being processed daily by search engines. What are you trying to achieve by continuously resubmitting the sitemap?
The site is indexed, correct? And when you look at crawl stats it is showing the site being crawled on some semi regular basis? Google does not process your sitemap every day.
Hope that helps,
Robert
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Sitemap.xml strategy for site with thousands of pages
I have a client that has a HUGE website with thousands of product pages. We don't currently have a sitemap.xml because it would take so much power to map the sitemap. I have thought about creating a sitemap for the key pages on the website - but didn't want to hurt the SEO on the thousands of product pages. If you have a sitemap.xml that only has some of the pages on your site - will it negatively impact the other pages, that Google has indexed - but are not listed on the sitemap.xml.
Technical SEO | | jerrico10 -
General questions about implementing hreflang using XML sitemap
I created another thread regarding hreflang sitemaps. However, this one is more general and doesn't cover multiple sitemaps for different localizations so I think it's reasonable creating a new thread. We are trying to implement hreflang using XML sitemap. We have localized content for a few countries, but only 1/3 of the content is 'duplicate' localized content. How should this be presented in the sitemap? Can we have some urls with hreflang-tags and some without? Also, where should this be located? In the usual sitemap file at site.com/sitemap.xml or should we create a different sitemap site.com/hreflang.xml where we just paste all hreflang-info? And if it should be in /hreflang.xml - can we have the same URL twice (in both current sitemap and hreflang sitemap)?
Technical SEO | | Telsenome0 -
Image Sitemap
I currently use a program to create our sitemap (xml). It doesn't offer creating an mage sitemaps. Can someone suggest a program that would create an image sitemap? Thanks.
Technical SEO | | Kdruckenbrod0 -
Sitemap issue
How can I create XML as well as HTML sitemaps for my website (both eCommerce and non - eCommerce )Is there any script or tool that helps me making perfect sitemapPlease suggest
Technical SEO | | Obbserv0 -
About how google works
Hey guyz,
Technical SEO | | atakala
I want to ask a basic question. If I search for Larry Page lets say.
I think google look for it's index for word larry and page distinct.
And mix it up. But the question ;
Can google show a result which only Larry exist on the page but any of the synonym or the stem of the Page not exist .
If it can happen how this page can be showned in larry page query. Thank you.0 -
Add selective URLs to an XML Sitemap
Hi! Our website has a very large no of pages. I am looking to create an XML Sitemap that contains only the most important pages (category pages etc). However, on crawling the website in a tool like Xenu (the others have a 500 page limit), I am unable to control which pages get added to the XML Sitemap, and which ones get excluded. Essentially, I only want pages that are upto 4 clicks away from my homepage to show up in the XML Sitemap. How should I create an XML sitemap, and at the same time control which pages of my site I add to it (category pages), and which ones I remove (product pages etc). Thanks in advance! Apurv
Technical SEO | | AB_Newbie0 -
Should I consider webmaster tools links and linked pages ratio to remove unnatural links?
I don't know this is a suitable place for post this question. Anyway I have done it. According to the Google webmaster tools, Links to your site page. My blog has considerable amount of links, from linked pages (from certain domain names). For an instance please refer following screenshot. When I am removing unnatural links, should I consider these, links from linked pages ratio? Almost all of these sites are social bookmarking sites. When I publish a new bookmark on those sites, they automatically add a homepage link. As a result of that, I got a huge number of home page links from linked pages. What is your recommendation? Thanks! webmaster.png web_master_tools.png
Technical SEO | | Godad0 -
Sitemaps
Hi, I have doubt using sitemaps My web page is a news we page and we have thousands of articles in every section. For example we have an area that is called technology We have articles since 1999!! So the question is how can Make googl robot index them? Months ago when you enter the section technology we used to have a paginator without limits, but we notice that this query consume a lot of CPU per user every time was clicked. So we decide to limit to 10 pages with 1 records. Now it works great BUT I can see in google webmaster tools that our index decreased dramatically The answer is very easy, the bot doesn't have a way to get older technoly news articles because we limit he query to 150 records total Well, the Questin is how can I fix this? Options: 1) leave the query without limits 2) create a new button " all tech news" with a different query without a limit but paginated with (for example) 200 records each page 3) Create a sitemap that contain all the tech articles Any idea? Really thanks.
Technical SEO | | informatica8100