Some URLs were not accessible to Googlebot due to an HTTP status error.
-
Hello I'm a seo newbie and some help from the community here would be greatly appreciated.
I have submitted the sitemap of my website in google webmasters tools and now I got this warning:
"When we tested a sample of the URLs from your Sitemap, we found that some URLs were not accessible to Googlebot due to an HTTP status error. All accessible URLs will still be submitted."
How do I fix this? What should I do?
Many thanks in advance.
-
You need to confirm that the URLs are in fact 100% of your URLs going into the site map are accessible.
if it's a big issue in a big site send me the URL in a private message I will use deep crawl to create a XML sitemap for you. The screaming frog tool is excellent as well though does performance well with extremely large sites.
check your robots.txt file this so great tool if in case you have more than one (it happens)
http://www.internetmarketingninjas.com/seo-tools/robots-txt-generator/
or
http://tools.seochat.com/tools/robots-txt-validator/
so many great free tools are found right here http://tools.seochat.com/tools/
It could be a number of things although it could be Google being finicky. Run the site through Moz crawler, use feedthebot.com using "tools SEO" or download the free version of http://www.screamingfrog.co.uk/seo-spider/ this will tell you if there is an issue. If your site is static you can even create an alternate site map with screaming frog if your site is large use deep crawl or Moz analytics
be certain there are no sitemaps redirecting to each other so no redirects going from the old site map to the new site map. Make certain that the site map is in an XML format e.g. http://example.com/sitemap.xml or if in a different format like https://example.com/sitemap_index.xml make sure the proper format That resolves when you look at the site map is what is going into Webmaster tools. Be certain the site map does not contain over 500 URLs per the site map so example.com/sitemap1.xml and so on keep numbering them appropriately. sometimes Google is overloaded and does not seem to like to play well with certain site maps or the site map is maybe not generating very well on the server and that is fixed later on. If this is a long-term problem speak to your host or developer. My recommendation is if you've done everything I have talked about that you attempt to submit is the sitemap to to Webmaster tools or simply build a new sitemap and submit that.
so if worse comes to worse take the screaming frog and use this URL to send it to Google
http://www.google.com/submityourcontent/business-owner/
I hope that helps,
Thomas
-
Hi, It looks like you have url's placed in your sitemap that have an HTTP status error. You can search for the urls and remove them from your sitemap or make sure they have the right status. Does it say which status error? And does it say which url's? Did you check those url's?When you use Screaming frog spider tool (free), you can search for status error's this is an easy way to find these url's.
Grtz, Leonie
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Sitemap error in Webmaster tools - 409 error (conflict)
Hey guys, I'm getting this weird error when I submit my sitemap to Google. It says I'm getting a 409 error in my post-sitemap.xml file (https://cleargear.com/post-sitemap.xml). But when I check it, it looks totally fine. I am using YoastSEO to generate the sitemap.xml file. Has anyone else experienced this? Is this a big deal? If so, Does anyone know how to fix? Thanks EwTswL4
Technical SEO | | Extima-Christian0 -
Clean URL vs. Parameter URL and Using Canonical URL...That's a Mouthfull!
Hi Everyone, I a currently migrating a Magento site over to Shopify Plus and have a question about best practices for using the canonical URL. There is a competitor that I believe is not doing it the correct way, so I want to make sure my way is the better choice. With 'Vendor Pages' in Shopify, they show up looking like: https://www.campusprotein.com/collections/vendors?q=Cellucor. Not as clean. Problem is that Shopify also creates https://www.campusprotein.com/collections/cellucor. Same products, same page, just a different more clean URL. I am seeing both indexed in Google. What I want to do is basically create a canonical URL from the URL with the parameter that points to the clean URL. The two pages are very similar. The only difference is that the clean URL page has some additional content at the top of the page. I would say the two pages are 90% the same. Do you see any issue with that?
Technical SEO | | vetofunk0 -
URL Parameters as pagination
Hi guys, due to some changes to our category pages our paginated urls will change so they will look like this: ...category/bagger/2?q=Bagger&startDate=26.06.2017&endDate=27.06.2017 You see they include a query parameter as well as a start and end date which will change daily. All URLs with pagination are on noindex/follow. I am worrying that the products which are linked from the category pages will not get crawled well when the URLs on which they are linked from change on a daily basis. Do you have some experience with this? Are there other things we need to worry about with these pagination URLs? cheers
Technical SEO | | JKMarketing0 -
Google webmaster tools says access denied error 403
Hi, this keeps on happening, just check early today and it tells me i have access denied and 403 errors I have this from time to time in my google webmaster tools and i have checked the pages and they work properly, so i am puzzled why this has happened. I have contacted my hosting company who have said there is not a problem but there must be a problem somewhere which could affect my site rankings. can anyone let me know what this could be please. i work in joomla | parenting-magazine | 403 | 8/10/13 |
Technical SEO | | ClaireH-184886
| | 2 | personal-finance-money-advice | 403 | 8/10/13 |
| | 3 | 201308081607/emmerdale/emmerdale-chas-confronts-cameron-over-affair-with-debbie | 403 | 8/10/13 |
| | 4 | 201308081606/emmerdale/emmerdale-declan-gets-a-visit-from-the-police | 403 | 8/10/13 |
| | 5 | 201308081608/emmerdale/emmerdale-cameron-debbie-affair-is-out-in-the-open | 403 | 8/10/13 |
| | 6 | 201308081614/uk-holiday-news/visitscotland-launch-campaign-to-boost-tourism | 403 | 8/10/13 |
| | 7 | dog-advice/training-your-puppy-a-beginners-guide | 403 | 8/10/13 |
| | 8 | gadgets/hp-envy-13-laptop-review | 403 | 8/10/13 |
| | 9 | gadget-talk/everyday-smartphone-gadgets-which-could-revolutionise-your-life | 403 | 8/10/13 |
| | 10 | news-gadgets/the-htc-one-mobile-phone-review | 403 | 8/10/13 |
| | 11 | gadget-talk/five-iphone-apps-for-home-improvement | 403 | 8/10/13 |
| | 12 | gadget-talk/are-android-apps-useful-for-business-success | 403 | 8/10/13 |
| | 13 | gadget-talk/television-gadgets-the-future-of-television-is-coming | 403 | 8/10/13 | | | |0 -
Do keywords in url parameter count?
I have a client who is on an older ecommerce platform that does not allow url rewrites in anyway. It would cost a ton of money to custom dev a solution. Anyways right now they have set up a parameter on their product urls to at least get the keyword in there. My question is, will this keyword actually be counted since it is in a parameter? An example url is http://domain.com/Catalog.aspx?Level1=01&Level2=02&C=Product-name-here Does this 'product-name-here' count as having the keyword in the url according to google?
Technical SEO | | webfeatseo0 -
Where to place your brandname in your URL?
Hello everybody! Quick and short question: What is better when you want to rank for your your brandname? www.jobsbrandname.com or www.brandnamejobs.com I think for SEO it's better to use the last one but marketing has the wish to use the first one. Thanks for your responce!
Technical SEO | | ltom0 -
Are URL's with trailing slash seen as two different URLs
Hello, http://www.example.com and http://ww.example.com/ Are these seen as two different URL's ? Just as with www or non www ? Or it doesn't make any difference ?
Technical SEO | | seoug_20050