We are still seeing duplicate content on SEOmoz even though we have marked those pages as "noindex, follow." Any ideas why?
-
We have many pages on our website that have been set to "no index, follow." However, SEOmoz is indexing them as duplicate content. Why is that?
-
Hi Gary,
Great answer from Daniel.
One thing that you can do is to create a list of noindexed pages in excel, then add all pages identified by SEOmoz as duplicates and run a simple comparison in excel. This will identify any pages that do not match. You will easily see whether the new pages in the report can be ignored.
There is already a feature request in the works with the SEOmoz engineering team which will enable us to "turn off" pages that can be ignored (like those that are already noindexed). In the meantime, keeping track of the pages you can ignore is probably the best option.
You can keep track of progress by following updates on the Feature Request here.
Hope that helps,
Sha
-
Go to Google and search site:yourdomain.com and see if the pages in question come up. If so, Google has indexed them. If not, Google has not indexed them. Like SEOMoz, Google can crawl any page. Doesn't mean they will index the page. If you have noindexed a page, it should not be indexed by Google and should not be problematic for you.
-
So, it indexes issues that Google does see and doesn't see. How do we differentiate between the two?
Additionally, what would be some suggestions as to what we should do?
-
SEOMoz is not a search engine index, it uses a crawler. If those pages are not blocked by the robots.txt file, then SEOMoz will crawl them. They ignore the noindex tag because they don't index anything. Search engines will honor the noindex tag and not index a page if you specify with the robots meta tag. However, to remove pages from the crawl, disallow them in the robots.txt.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Does this type of writing follow the "original content" criterion of structured data?
Hi!' So, in Google's general guideline for structured data, it's stated that the webmasters must "provide original content that you or your users have generated." If I were to write an article about post similar to stuff like "how to get a driver's license" or "how to apply for an accounting license", which requires looking up information from official and non-official sources. After researching, I compiled the information I found and wrote a few blog posts. Are these considered original content? Can I apply structured data to these posts without Google penalizing them? Thanks!
Technical SEO | | EverettChen0 -
Quick Fix to "Duplicate page without canonical tag"?
When we pull up Google Search Console, in the Index Coverage section, under the category of Excluded, there is a sub-category called ‘Duplicate page without canonical tag’. The majority of the 665 pages in that section are from a test environment. If we were to include in the robots.txt file, a wildcard to cover every URL that started with the particular root URL ("www.domain.com/host/"), could we eliminate the majority of these errors? That solution is not one of the 5 or 6 recommended solutions that the Google Search Console Help section text suggests. It seems like a simple effective solution. Are we missing something?
Technical SEO | | CREW-MARKETING1 -
Possible duplicate content issue with my Blog and archive pages . Any help greatly appreciated
Dear Mozzers, I have been looking at my news section on my eCommerce site and I think I may have a duplicate content issue and wanted some advice on whether I do and if so , how best I handle this. http://www.website.co.uk/news
Technical SEO | | PeteC12
http://www.website.co.uk/news/page:1
http://www.website.co.uk/news/page:2
http://www.website.co.uk/news/page:3
http://www.website.co.uk/news/limit:9999 (This is show all) I also have the ability of showing articles by month : http://www.website.co.uk/news/archive/2015/04 (April)
http://www.website.co.uk/news/archive/2015/03 (March)
http://www.website.co.uk/news/archive/2015/02 (Feb)
http://www.website.co.uk/news/archive/2015/01 (Jan) I am wondering if there's a duplicate issue here or not given that I also articles by month as well and if so how best I handle this.? I already do pagination on my news pages (page 1 , page 2) by using rel=next and rel=Prev but I don't have an canconical or anything as yet. I enclose a couple of links if this would help and would appreciate if someone could take a browse. I have a View All link on my homepage for for all news items - http://goo.gl/JPPIvQ I which have a different urls - March 2015 Articles - http://goo.gl/0O1wYD and April 2015 articles - http://goo.gl/GdW2oK On another note, These articles are also linked to from the relevant category landing pages on my website to help with SEO. I have not used H tags on the article links in my landing pages , just displaying the weblink back to the news article.I've done this to try and improve the PR and rankings of my landing pages. Just wondered if anyone has any comments as to whether thats a good or bad idea and whether I could improve it in any way - An example is here (scroll down the page to the pressure washing guides) - http://goo.gl/nnRE49 Thanks Pete0 -
Big page of clients - links to individual client pages with light content - not sure if canonical or no-follow - HELP
Not sure what best practice here is: http://www.5wpr.com/clients/ Is this is a situation where I'm best off adding canonical tags back to the main clients page, or to the practice area each client falls under? No-following all these links and adding canonical? No-follow/No-index all client pages? need some advice here...
Technical SEO | | simplycary0 -
How do I fix issue regarding near duplicate pages on website associated to city OR local pages?
I am working on one e-commerce website where we have added 300+ pages to target different local cities in USA. We have added quite different paragraphs on 100+ pages to remove internal duplicate issue and save our website from Panda penalty. You can visit following page to know more about it. And, We have added unique paragraphs on few pages. But, I have big concerns with other elements which are available on page like Banner Gallery, Front Banner, Tool and few other attributes which are commonly available on each pages exclude 4 to 5 sentence paragraph. I have compiled one XML sitemap with all local pages and submitted to Google webmaster tools since 1st June 2013. But, I can see only 1 indexed page by Google on Google webmaster tools. http://www.bannerbuzz.com/local http://www.bannerbuzz.com/local/US/Alabama/Vinyl-Banners http://www.bannerbuzz.com/local/MO/Kansas-City/Vinyl-Banners and so on... Can anyone suggest me best solution for it?
Technical SEO | | CommercePundit0 -
Duplicate pages
Hi Can anyone tell me why SEO MOZ thinks these paes are duplicates when they're clearly not? Thanks very much Kate http://www.katetooncopywriter.com.au/how-to-be-a-freelance-copywriter/picture-1-58/ http://www.katetooncopywriter.com.au/portfolio/clients/other/ http://www.katetooncopywriter.com.au/portfolio/clients/travel/ http://www.katetooncopywriter.com.au/webservices/what-i-do/blog-copywriter/
Technical SEO | | ToonyWoony0 -
Tags causing Duplicate page content?
I was looking through the 'Duplicate Page Content' and Too Many On-Page Link' errors and they all seem to be linked to the 'Tags' on my blog pages. Is this really a problem and if so how should I be using tags properly to get the best SEO rewards?
Technical SEO | | zapprabbit1 -
Site Navigation leads to "Too Many On-Page Links" warning
I run an ecommerce site with close to 2000 products. Nearly every page in the catalog has a too many on-page links error because of the navigation sidebar, which has several flyout layers of nested links. What can/should I do about this? Will it affect my rankings at all? Thanks
Technical SEO | | AmericanOutlets0