Update in Moz spider/tools?? Flagging duplicate content / ignoring canonical
-
Hi all,
Has there been an update in the SEOmoz crawling software?
We now have thousands of dupe content/page title warnings for paginated product page URLs that have correctly formatted canonicals.
e.g.
http://www.woolovers.com/british-wool/mens/tweed-green/wool-countryman-suede-patch-sweater.aspx
... has following pages with identical content that have been flagged:
http://www.woolovers.com/british-wool/mens/olive-green/wool-countryman-suede-patch-sweater.aspx?p=true&rspage=4
..plus 4 more URL's.
But they all have canonical set. There's even a notice at the bottom of report that tells us there's a canonical set to http://www.woolovers.com/british-wool/mens/tweed-green/wool-countryman-suede-patch-sweater.aspx
What gives, SEOmoz ??
Thanks
Michael
-
Hey Lawrence,
Campaigns have a 95% tolerance for duplicate content. This includes all the source code on the page and not just the viewable text. So if a URL is at least 95% similar in code and content to another URL, this warning will appear.
You can run your own tests using this tool: http://www.webconfs.com/similar-page-checker.php
We don't know what standard Google uses, but it's safe to say they are a bit more sophisticated than us - so you might be okay in this regard as long as you have a couple hundred words of unique text and some unique coding per page. Google won't say how much duplicate content is too much, so we like to be better safe than sorry.
I hope this help. Let me know if you need further assistance.
-Chiaryn
-
Hi Chiaryn,
Thanks for reply and explanation. The different colour-specific pages e.g. Tweed Green and Olive Green have some different content but it's nothing like enough in cases of two greens, two blues etc. as we simplify colour names for search so when there is an Olive and a Tweed Green they both end up having 'Green' as variable in page title, H1 etc. Will fix this.
Do you think the reviews at the bottom of the pages will also trigger dupe content warning? i.e. even if we make all other on-page elements unique for each colour url? (page title, H1, H2, prod description etc) The reviews are quite extensive and are the same on all the separate colour specific product page versions of each style and was thinking today whether we should remove them from these colour product pages (OR perhaps let the colour product pages have their OWN reviews)
http://www.woolovers.com/british-wool/mens/tweed-green/wool-countryman-suede-patch-sweater.aspx
Thanks again
-
Oh, brilliant (re: "See more" aspect) Thanks for the info. Will let you how we tackle this and the repercussions (!) and look forward to hearing how you get on also!
-
Hi Michael,
Thanks for writing in. I already emailed you in response to the ticket you sent in to the Help Desk, but I will copy my answer here for you review.
--
I looked into your campaign and it seems that this is happening because of where your canonical tags are pointing. These pages are considered duplicates because their canonical tags point to different URLs. For example, http://www.woolovers.com/british-wool/mens/tweed-green/wool-countryman-suede-patch-sweater.aspx is considered a duplicate of http://www.woolovers.com/british-wool/mens/olive-green/wool-countryman-suede-patch-sweater.aspx?p=true&rspage=4 because the canonical tag for the first page is http://www.woolovers.com/british-wool/mens/tweed-green/wool-countryman-suede-patch-sweater.aspx while the canonical for the second URL ishttp://www.woolovers.com/british-wool/mens/olive-green/wool-countryman-suede-patch-sweater.aspx, with one URL showing tweed-green and the other showing olive-green.
Since the canonical tags point to different URLs it is assumed that http://www.woolovers.com/british-wool/mens/tweed-green/wool-countryman-suede-patch-sweater.aspx and http://www.woolovers.com/british-wool/mens/olive-green/wool-countryman-suede-patch-sweater.aspx are likely to be duplicates themselves.
Here is how our system interprets duplicate content vs. rel canonical:
Assuming A, B, C, and D are all duplicates,
If A references B as the canonical, then they are not considered duplicates
If A and B both reference C as canonical, A and B are not considered duplicates of each other
If A references C as a canonical, A and B are considered duplicated
If A references C as canonical, B references D, then A and B are considered duplicates
The examples you've provided actually fall into the fourth example I've listed above.I hope this clears things up. Please let me know if you have any other questions.
--
-Chiaryn
-
We use the "See more" script on our sites, and from what I understand, at least from other Mozzers, this is an okay practice. http://www.seomoz.org/q/using-more-info-javascript-toggledisplay-tag-for-more-info-text
We also use the rel="prev" and rel="next" to some success, but I can't comment on how that's functioning canonical-wise, because IT WAS DROPPED from our latest redesign and is going to be added to our client's website in the latest release. Oye.
I'd love to hear how this works out for you. There are some really great Mozzers on here with loads of experience about canonical tags and duplicate page issues. Can't wait to see what they have to contribute.
-
Hi there,
Thanks for your response.
It's not product page A being seen as a duplicate of product page B etc, but several versions of product A seen as duplicate due to pagination, stemming from reviews for the products that span several pages, so making the rest of the content, titles etc different other than the (crawlable) reviews isn't really an option.
Will look more into "noindex, follow" tags in pagination.
We could have a View All page for indexing showing all reviews (with lots of scrolling!) , with the paginated versions canonicalized to that version (could still serve the paginated version of product page from site navigation perhaps with "noindex, follow" meta tag) Text doesn’t take long to load and this approach would consolidate the review content.
http://googlewebmastercentral.blogspot.co.uk/2011/09/view-all-in-search-results.html
Other option is to use rel=”prev” and rel=”next” implementation which shows Google the relationship between the pages (not sure if it will still be flagged as dupe content in SEOmoz though! Depends if they follow the tag). This way individual pages might get indexed (not sure if that's a good thing?!) perhaps if there's something in a review from (say) page 5 of the product reviews.
http://googlewebmastercentral.blogspot.co.uk/2011/09/pagination-with-relnext-and-relprev.html
Ideally I'd like to implement all reviews on one page and hide them with a facebook-style 'See more' function. Not sure if that counts as hiding content? Will look into this.
-
Hi Michael,
Not sure if this helps you out at all, but I found this about the canonicals and SEOMoz crawl report in a previous Q http://mz.cm/11erRj6:
As far as the SEOmoz crawl reports go, not that setting a canonical won't stop these pages being reported as duplicate content.
From the help:
"Keep in mind that that canonicals will stop the pages from ranking against each other, but they will still show up as duplicate content from a UI perspective, so we will still count them as duplicate."
I have the same issues on my accounts. I'm focusing on making the pages content as unique as possible, or using the "noindex, follow" meta tags to see if that makes a difference.
I know you may have a lot of pages on your website, but perhaps writing short descriptions on your products would help. It might be worthwhile, but completely understandable that it may be a huge undertaking if you have hundreds or thousands of pages.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Duplicate content report - question on best practice
Hello all, New to MOZ Pro and SEO - so lots to get my head round! I’m working through the Duplicate Content section of the Crawl report and am not sure what the best practice is for my situation. Background: We are a reference guide for luxury hotels around the world, but the hotels that are featured on the site vary year on year. When we add a new hotel page, it sets up the url as ourwebsite.com/continent/country/regionORcity/hotel. When the hotels come off, I redirect their URL to the country or region where we have other hotels. Example: http://www.johansens.com/europe/switzerland/zermatt/ The hotel in Zermatt has come off the site, showing 0 results on this landing page. Question: My duplicate content report is showing a number of these regional pages that are displaying the copy “0 places - Region’ because the hotel has come off, but the landing page is still live. Should I redirect the regional page back to the main country page? And then if I add a new hotel to the site from that region in the future, simply remove the redirect? Should I also delete the page? Any tips would be much appreciated!
Moz Pro | | CN_Johansens0 -
Interested in testing a new Moz product for Content Strategists?
We're building a new product for content professionals that helps with website auditing, content performance measurement and content discovery/curation. If you're a content strategist, blogger, editor, copywriter, SEO with a content focus or social media manager that curates content we'd love to hear your thoughts. Interested in a sneak peak of what's cooking? Either respond to this post or send me an email at jay@moz.com. Be sure to include your job function and the type of company your work for. Thanks all!
Moz Pro | | JayLeary5 -
My moz campaing not update completely
I have create new account and campaign last week, till now my moz campaign not update like: keywords ranking, competitor websites links etc. Why it is so please update me.
Moz Pro | | Surabhi_Dewra0 -
How to handle subdirectories in Moz and Webmaster Tools?
I have 3 websites: http://www.abc.com (English pages) http://www.abc.com/fr (French) http://www.abc.com/de (German) Webmaster Tools In Google Analytics: I created an Account called ABC and created 4 properties with relative filters ABC Default (No filter) ABC English (filter to exclude /fr, /de) ABC French (filter to include /fr) ABC German (filter to include /de) In webmaster tools, I wish to add 3 sites http://www.abc.com http://www.abc.com/fr http://www.abc.com/de How do I validate each url with Google analytics, considering I just want the specific language for folder. How do I validate the domain name http://www.abc.com to always ensure the inclusion of the default english data? MOZ Do I need to create 3 campaigns (one per language) and if yes, again how do I handle http://www.abc.com which should exclude /fr and /de? Thank you
Moz Pro | | seo12120 -
Can't figure out why some of my pages are duplicate content
Within the crawl diagnostics area I'm getting duplicate page content issues on several pages. I don't know why, would anyone be able to tell me how these links are duplicate so I can fix them? http://www.sagenews.ca/Column.asp?id=3010 http://www.sagenews.ca/Column.asp?id=2808 http://www.sagenews.ca/Column.asp?id=2998 http://www.sagenews.ca/Column.asp?id=2837 http://www.sagenews.ca/Column.asp?id=2981
Moz Pro | | INMCA0 -
Canonical for Mobile
Hi Guys, I am curious why in SEOMoz, our mobile site is showing to have the canonical tags used on the desktop site but when you double check the code of the mobile website it is showing m.domain.com Any thoughts on why we are seeing this? Also is there any lag in the code updates being reported through the SEOmoz toolset? Thanks for all your help! Cheers,
Moz Pro | | lwalker0 -
Which tools do not use backlinking and/or tagging?
I am new to SEOmoz and was wondering if any, which tools on SEOmoz does not use or require backlinking and/or tagging?
Moz Pro | | aschraegle0 -
A tool to submit websites in directories
Hello I am looking for a tool to help me to submit websites in directories, something like the yooda tool. http://www.yooda.com/outils_referencement/submit_center_yooda/ This tool seems good no? do you offer something similar at seomoz? or where could I find some similar tools and in which languages is it available?
Moz Pro | | bigtimeseo2