How can I check if the FOLLOW,NOINDEX tag is working?
-
Hi everyone!
After reading about pagination practices, a few days ago we introduced the <meta name="robots" content="FOLLOW,NOINDEX" /> tag, to prevent duplicate content.
You can find an example below:
http://www.inmonova.com/en/properties?page=2
I have been checking yahoo site explorer and result pages still get indexed. My question is:
Am I doing something wrong? Is the code incorrect (follow,noindex - noindex,follow)? Or does it just take some time to have effect?
Thanks in advance.
-
Excactly what I was looking for. Thanks.
-
There are sites out there like http://www.noindexchecker.com/ that will help you double check your work
-
Thank you very much for your quick responses!
I will wait patiently to see those dulicates drop down
-
Daniel is correct. You just need to be more patient.
Websites are crawled in pieces. The crawler can grab 10% of your site each day. It depends upon the popularity of your site, the popularity of the content within your site, how well your site is linked, along with other factors.
You have done all you can for now. You will see your pages start disappearing from search results.
-
The code is correct and I see that there are 0 links to that results page - the only thing I can think of is the search engines indexed the page before you added the tag!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Important category pages that can and should be found in SERP but can not be reached by navigating on the webshop itself
Hi, On a webshop we are optimizing, the main navigation consists of the 5 main categories to which all of the products can be assigned. However, the main tabs in the navigation just activate a drop down with all of the subcategories. For example: the tab in the navigation is 'Garden equipment' and when you click on this tab, the drop down is shown with subcategories like 'Lawn mowers', 'Leaf blowers' and so on. Now, the page 'Garden equipment' is one of the main category pages and we want this page to rank of course. This shouldn't be a problem, since there is a separate URL for this page that can be indexed and that can be reached through internal links on the website. However, this page can not be reached when a visitor initially comes on the homepage of the webshop, since the tab in the navigation isn't clickable. This page will only be reached when a subcategory is selected, and then when the visitor goes back to the category page through the breadcrumb or through an internal link. Is it a problem that these important overview category pages can not be reached immediately? Thanks.
Intermediate & Advanced SEO | | Mat_C0 -
Can anyone help me figure out these sitelinks?
My company is Squatty Potty (yes, of magic unicorn fame) and I recently redid our website's navigation. We're overhauling it currently to rebuild the whole thing, but what is there should give a good idea of site hierarchy to Google I would think. The funny thing is, when you Google [squatty potty website] we do have sitelinks. But when you Google just [squatty potty] we don't. Any ideas on why sitelinks would appear on one search but not the other? I see they appear with [squatty potty logo] as well. I can't figure out how to get them to appear for my brand name search, any help appreciated!
Intermediate & Advanced SEO | | DanDeceuster0 -
How to check if the page is indexable for SEs?
Hi, I'm building the extension for Chrome, which should show me the status of the indexability of the page I'm on. So, I need to know all the methods to check if the page has the potential to be crawled and indexed by a Search Engines. I've come up with a few methods: Check the URL in robots.txt file (if it's not disallowed) Check page metas (if there are not noindex meta) Check if page is the same for unregistered users (for those pages only available for registered users of the site) Are there any more methods to check if a particular page is indexable (or not closed for indexation) by Search Engines? Thanks in advance!
Intermediate & Advanced SEO | | boostaman0 -
"noindex, follow" or "robots.txt" for thin content pages
Does anyone have any testing evidence what is better to use for pages with thin content, yet important pages to keep on a website? I am referring to content shared across multiple websites (such as e-commerce, real estate etc). Imagine a website with 300 high quality pages indexed and 5,000 thin product type pages, which are pages that would not generate relevant search traffic. Question goes: Does the interlinking value achieved by "noindex, follow" outweigh the negative of Google having to crawl all those "noindex" pages? With robots.txt one has Google's crawling focus on just the important pages that are indexed and that may give ranking a boost. Any experiments with insight to this would be great. I do get the story about "make the pages unique", "get customer reviews and comments" etc....but the above question is the important question here.
Intermediate & Advanced SEO | | khi50 -
Noindex search pages?
Is it best to noindex search results pages, exclude them using robots.txt, or both?
Intermediate & Advanced SEO | | YairSpolter0 -
Meta tag description Usage
Do i wanna put meta tags as separate description that is not in the particular web page ,Normally i put meta description as 155 character from first paragraph of the web page .so do i need put a unique meta description ?
Intermediate & Advanced SEO | | innofidelity0 -
Are tags an issue in SEO
SEOMoz saw that my tags were duplicate pages. Are tags a serious issue in SEO? Should I remove it entirely to prevent the duplicate pages?
Intermediate & Advanced SEO | | visualartistics0 -
No index, follow vs. canonical url
We have a site that consists almost entirely as a directory of videos. Example here: http://realtree.tv/channels/realtreeoutdoorsclassics We're trying to figure out the best way to handle pagination and utility features such as sort for most recent, most viewed, etc. We've been reading countless articles on this topic, but so far have been unable to determine what might be considered the industry standard. Two solutions seem to stand out... Using the canonical url on all the sorted and paginated pages. However, after reading many blog posts, it seems that you should NEVER use the canonical url to solve the issue of paginated, and thus duplicated content because the search bots will never crawl past the first page leaving many results not in the index. (We are considering ruling this method out.) Another solution seems to be using the meta tag for noindex, follow so that a search engine like Google will crawl your directory pages but not add them to the index themselves. All links are followed so content is crawled and any passing link juice remains unchanged. However, I did see a few articles skeptical of this solution as well saying that there are always better alternatives, or that there is no verification that search engines obey this meta tag. This has placed some doubt in our minds. I was hoping to get some expert advice on these methods as it would pertain to our site. Thank you.
Intermediate & Advanced SEO | | grayloon0