Sitemap_index.xml = noindex,follow
-
I was running a rapport with Sreaming Frog SEO Spider and i saw:
(Tab) Directives > NOindex :
https://compleetverkleed.nl/sitemap_index.xml/ is set on X-Robots-Tag 1 > noindex,follow
Does this mean my sitemap isn't indexed?
If anyone has some more tips for our website, feel free to give some suggestions
(Website is far from complete)
-
Top, thanks!
-
Hi There
I don't think you need to worry about the sitemap being indexed or not - it's an XML sitemap, not an HTML page users will need to find. It's accessible to Google, and they will use it to crawl the site. Have you submitted the XML sitemap to webmaster tools? If so, make sure it's free of errors and you should be all set!
-
Hi Patrick,
Thanks for the support. I'm only wondering how to fix this problem with a Wordpress website:
https://www.compleetverkleed.nl/sitemap_index.xml/ There should not be a trailing at the end of this URL. This needs to be fixed as soon as possible.
It was set in our footer and linked to the sitemap, and i removed the "/". But in Screaming frog i still see:
https://compleetverkleed.nl/sitemap_index.xml is set on X-Robots-Tag 1 > noindex,followI would also make sure that your non www. sitemap redirects to https://www.compleetverkleed.nl/sitemap_index.xml. Where can i fix this?
This should clear up your problem. Make sure this URL reflects in your Google and Bing Webmaster Tools. Like this? (Google Webmaster tools copy)
|
# Sitemap
Type Verwerkt Problemen Items Verzonden Geïndexeerd --- --- --- --- --- --- --- --- --- 1 /sitemap_index.xml Sitemapindex 19 mei 2015 - In behandeling In behandeling -1 van 1 -
Hi there
This doesn't appear to be your sitemap. Your sitemap lives at:
https://www.compleetverkleed.nl/sitemap_index.xml
However, this works as well:
https://www.compleetverkleed.nl/sitemap_index.xml/
There should not be a trailing at the end of this URL. This needs to be fixed as soon as possible.
I would also make sure that your non www. sitemap redirects to https://www.compleetverkleed.nl/sitemap_index.xml.
This should clear up your problem. Make sure this URL reflects in your Google and Bing Webmaster Tools.
Let me know if this helps - good luck!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Do Google Penalties Always Follow a Redirects to New Domains?
I have a couple sites that were penalized by Google for hosting content that made Google look bad. After a major newspaper showcased what was going on they suddenly took a major hit as if someone at Google flipped a switch and told their system not to rank the content for anything other than their brand names. The article made Google look bad because the newspaper highlighted a lot of unverified user generated accusations the reporters assumed not to be true in the context of "these accusations are mostly false, but they still show up on the first page when people search Google." I was thinking one way to fight this would simply be to host the content at a different domain, but I am concerned about the new domain being penalized as well. I don't want to completely shut down all of the original sites because some of them have brand recognition. The oldest domain is 12 years old with backlinks from several news outlets which is why the content ranked so well, but after the penalty that is only the case on Bing. I've read various articles about this tactic. Some say that you will almost always pass the penalty to the new domain if you do a 301 redirect, but the penalties at issue in those articles were for things like buying links or other black hat tactics. This is somewhat different in that I wasn't doing anything black hat, they just decided not to let the site rank for political reasons. I was hoping that maybe that type of penalty wouldn't follow it, but right now I am leaning towards simply creating a second site to syndicate articles. It will need to attribute the articles to their sources though, so they will need either no followed links or possibly a redirection script that bots cannot follow. I would really like it if I could simply change the first site to its .net or .org equivalent and 301 everything though.
Technical SEO | | PostAlmostAnything0 -
Why xml generator is not detecting all my urls?
Hi Mozzers, After adding 3 new pages to example.com, when generating the xml sitemap, Iwasn't able to locate those 3 new url. This is the first time it is happening. I have checked the meta tags of these pages and they are fine. No meta robots setup! Any thoughts or idea why this is happening? how to fix this? Thanks!
Technical SEO | | Ideas-Money-Art0 -
Thesis Theme (Nofollow, noindex) Problem
Hi, Im using Thesis theme for one of my wordpress website, for some reason, some of my pages are 'noindex" and 'nofollow" even though i have these boxes unchecked Does anybody know the solution to that? Thanks
Technical SEO | | KentR0 -
Is a Rel="cacnonical" page bad for a google xml sitemap
Back in March 2011 this conversation happened. Rand: You don't want rel=canonicals. Duane: Only end state URL. That's the only thing I want in a sitemap.xml. We have a very tight threshold on how clean your sitemap needs to be. When people are learning about how to build sitemaps, it's really critical that they understand that this isn't something that you do once and forget about. This is an ongoing maintenance item, and it has a big impact on how Bing views your website. What we want is end state URLs and we want hyper-clean. We want only a couple of percentage points of error. Is this the same with Google?
Technical SEO | | DoRM0 -
Noindex Pages indexed
I'm having problem that gogole is index my search results pages even though i have added the "noindex" metatag. Is the best thing to block the robot from crawling that file using robots.txt?
Technical SEO | | Tedred0 -
Do I need an XML sitemap?
I have an established website that ranks well in Google. However, I have just noticed that no xml sitemap has been registered in Google webmaster tools, so the likelihood is that it hasn't been registered with the other search engines. However, there is an html sitemap listed on the website. Seeing as the website is already ranking well, do I still need to generate and submit an XML sitemap? Could there be any detriment to current rankings in doing so?
Technical SEO | | pugh0 -
Google SERPs and NoIndex directives.
We have pages that have been added to robots.txt as url patterns in DisAllow. Also, we have the meta noindex tags on the pages themselves. But we are finding the pages in index. I don't think they are higher up in the rankings and they don't have any descriptions, any previews or any cached pages. Why does Google show these pages? Could it be due to internal or external linking?
Technical SEO | | gaganc0 -
Does Google follow a text based URL with no anchor tags?
I am seeing with Webmaster Tools that Google is trying to follow the text based truncated URL from SuperPages despite the fact that they are not in an anchor tag. The net result for Google is a 404 error as they try to access pages that do not exist. has anyone seen this issue before and any suggestions on how to prevent these errors? A Superpage listing: The first link works fine, but the text based link shown below is cut off and as a result Google gets to a 404 page. http://swbd-out.superpages.com/webresults.htm?qkw=dr+hylton+lightman&qcat=web&y=0&x=0& Dr. Hylton Lightman, MD - Pediatrician, Allergist, - Far Rockaway ... Dr. Hylton Lightman, MD, Far Rockaway, NY, Rated 4/4 By Patients. 26 Reviews, Patients' Choice Award Winner, Phone Number & Practice Locations www.vitals.com/doctors/Dr_Hylton_Lightma... [Found on Bing]
Technical SEO | | irvingw0