Meta Robots Tag - What's it really mean?
-
I used on a handful of pages recently and noticed that they're still popping up in the Google search index. I'd like to keep these from appearing, so I figured I needed a directive statement with stronger semantic meaning.
From what I understand, is what I'm looking for. Using this will keep Google from not only crawling the page, but indexing the page, as well.
I decided to see what the official robotstxt.org website said about it, so I checked (link here):
the NOFOLLOW directive only applies to links on this page. It's entirely likely that a robot might find the same links on some other page without a NOFOLLOW (perhaps on some other site), and so still arrives at your undesired page.
So, is their explanation saying that the page itself will be indexed, but the content / links on it won't be followed / indexed? Let me hear your thoughts, mozzers.
-
You can find out your desired meta robots setting by asking yourself two questions:
noindex or index: do you want this particular page to show up as a listing in the Google results for a query? Yes: index, No: noindex.
nofollow or follow: do you want the links on this page to be crawled (followed) by the search engine robots, thereby allowing them access to the deeper pages it links to (even though that particular page might even be noindexed). Yes: follow, No: nofollow
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Hey guys, any advice on why my 'search analytics' in Google Webmaster tools is blank with no data?
I have added all the website versions into Google web master tools and I have no crawl errors. When I click on Search traffic these pages are blank - search analytics, & Mobile usability. And When I fetch as Google the status is constantly saying me 'temporarily unreachable' - any help would be greatly appreciated.
Reporting & Analytics | | HLAS0 -
Submitting an 'HTTPS' sitemap.xml to Bing
I have been trying to submit my sitemap to Bing [via their webmaster tools] for well over a week and it continues to report 'pending' My site is HTTPS and the sitemap is accepted by Google. I questioned Bing about this and got this response: To set your expectations, our Sitemap fetchers use a different pipeline and because of this, we cannot crawl Sitemaps in HTTPS format. We require that you submit an HTTP version of sitemap in order for Bing to properly crawl the file. Please go ahead and delete the current Sitemap and resubmit a new one in HTTP. Currently I don't and can't have a HTTP version of my site & sitemap and my developers are telling me that 3hrs worth of dev time will go into coming up with a work-around which I'm not sure I want to invest in [I have more important things to concentrate my spend on!]. Has anyone been faced with this problem and is there any quick/cheap alternative or do I just accept that Bing won't crawl my site until they update their end?!
Reporting & Analytics | | cityxplora.com0 -
Can't identify the real source of a goal because subdomains in GA
Hello, I have one domain for my website (www.domain.com) where I receive all the new traffic and a subdomain (billing.domain.com) for the billing system. Im tracking both in Analytics. I have a goal (new order) configured in the subdomain -billing system-. But when I want to track the source / medium of the goals (new orders) in billing.domain.com, always appears my main website (www.domain.com) as the referral. Now I added in Analytics: Admin > Property > Referral Exclusion List, the www.domain.com and domain.com. Do you think that now I can track the real referrer of the goals? Or should I do a more complex tracking setup with Tag Manager ( https://support.google.com/tagmanager/answer/6106951 ) to pass the session cookies from a subdomain to another one, just to identify the real source / medium of the visitor? Thank you 🙂
Reporting & Analytics | | estebanseo0 -
Why are there url's in my np Keywords in google analtyics?
I have been scratching my head at this one. On my not provided keywords, there is a url. Keyword is the primary dimension. It doesn't makes sense why there would be url keywords for pages with a different url. For example, the keywords for page www.xyz.com/abc np - /someotherurl.com Thanks for your insights in advance!
Reporting & Analytics | | konverge0 -
Robots.txt file issue.
Hi, Its my third thread here and i have created many like it on many webmaster communities.I know many pro are here so badly needs help. Robots.txt blocked 2k important URL's of my blogging site http://Muslim-academy.com/ Especially of my blog area which are bringing good number of visitors daily.My organic traffic declined from 1k daily to 350. I have removed the robots.txt file.Resubmitted existing Sitemap.Used all Fetch to index options and 50 URL submission option in Bing Webmaster Tool. What Can I do know to have these blocked URL's back in Google index? 1.Create a NEW sitemap and submit it again in Google webmaster and bing webmaster tool? 2.Bookmark,linkbuilding or share the URL's.I did a lot of bookmarking for blocked URL's. I fetch the list of blocked URLS Using BING WEBMASTER TOOLS.
Reporting & Analytics | | csfarnsworth0 -
Google Analytics Content Experiments don't deliver 50/50?
Our A/B test is actually delivering at about a 70/30 page view rate. 70% in favor of the original version and only 30% of the new. We are sending 100% of our traffic to this homepage test. Has anyone else experienced this? There seems to be a lot of folks experiencing this.....anyone know why?
Reporting & Analytics | | VistageSEO0 -
When I look at my SEOMOZ campaigns I see there are a lot of warnings in regards to missing Meta Tags Descriptions but they exist on a clien'ts wordpress site
when I look at my SEOMOZ campaigns I see there are a lot of warnings in regards to missing Meta Tags Descriptions but they exist on a clien'ts wordpress site
Reporting & Analytics | | Doug_Hay1 -
Will Bing/Google's engine index a page that has only been on social media?
Will Bing's engine index and rank a page that has only been seen on social media and has no inbound links? Will Google's? Are inbound links absolutely required to get a page indexed and ranking and getting traffic? If unknown, how would you go about testing this?
Reporting & Analytics | | SarahGoliger0