Duplicate URL errors when URL's are unique
-
Hi All,
I'm running through MOZ analytics site crawl report and it is showing numerous duplicate URL errors, but the URLs appear to be unique. I see that the majority of the URL's are the same, but shouldn't the different brands make them unique to one another?
http://www.sierratradingpost.com/clearance~1/clothing~d~5/tech-couture~b~33328/
http://www.sierratradingpost.com/clearance~1/clothing~d~5/zobha~b~3072/
Any ideas as to why these would be shown as duplicate URL errors?
-
There is long article on the dev blog how they determine whether pages are duplicates - check https://moz.com/devblog/near-duplicate-detection/ - it's quite technical stuff - but this is the part which might interest you:
"This leads to one of the questions we get asked a lot: Why do I see duplicate content warnings in the context of Custom Crawl for pages that I see as different. Ultimately, it’s always because of the same reason: because no dechroming is done, there is a small amount of unique content relative to the total content. One of the places where this crops up a lot is web stores, where there’s a large amount of chrome layout, but only a short product description associated with it."
Dechroming : removing things like navigation, footer, ..etc from the page (exact def. to be found in the article)
If you compare both pages - apart from the image & product title there isn't too much difference between them so the crawler sees only a very small % of content which is different and marks them as duplicates.
Dirk
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Duplicate content penalty
when moz crawls my site they say I have 2x the pages that I really have & they say I am being penalized for duplicate content. I know years ago I had my old domain resolve over to my new domain. Its the only thing that makes sense as to the duplicate content but would search engines really penalize me for that? It is technically only on 1 site. My business took a significant sales hit starting early July 2013, I know google did and algorithm update that did have SEO aspects. I need to resolve the problem so I can stay in business
On-Page Optimization | | cheaptubes0 -
Title Tag duplication.
Hi Guys/Gals, We do a lot of work in a very competitive space (personal injury) and are having an internal debate on the best way to implement title tags for new sites. We understand that keywords, title tags, etc., don't possess the power they once did, but we have yet to see conclusive proof of this in our space. The vast majority of competitors still rank very well for keyword focused content, title tags, etc., while having average link profiles and little content. We write a lot of content for our clients and want to know if someone can offer their opinion on the question that follows this example: "Top 5 Injuries Caused by T-Bone Collisions | Indiana Accident Lawyer" Would it seem repetitive or manipulative to construct title tags as shown, always placing "Indiana Accident Attorney" or "Indianapolis Accident Lawyer," or similar of at the end of each title tag? Thanks, gang!
On-Page Optimization | | Wayne760 -
Title tag and URL Optimization
Hello guys, Should the URL reflect the structure of the title of a webpage? This is the old title with the Url: 20mm O/D Black Polypropylene LSZH Flexible Conduit 100m Coil /Product/20mm-o-d-black-polypropylene-lszh-conduit-100m-coil/1352 I changed the keyword position and it looks like this: 20mm Flexible Conduit | O/D Black Polypropylene LSZH | 100m I kept the same Url for now, should I change that too? Thanks
On-Page Optimization | | PremioOscar0 -
What are your top tactics for boosting your site's Author Rank?
Mike Arneson has an excellent Mozinar where he shares some helpful Author Rank tactics. What specific tactics are you doing to boost the Author Rank of your site?
On-Page Optimization | | ProjectLabs1 -
Would adding a line break tag into the product name affect SEO ranking and Google's ability to read the entire title?
Our client would like to include a link break so that part of the product name always showed up on a second line. Would this affect how Google bots crawl the product name? Would it also affect how Google would show the product name in a search result page? Thanks!
On-Page Optimization | | BrandLabs0 -
Duplicate content - what to do?
Hi, We have a whole lot of articles on our site. In total 5232 actually. The web crawler tells me that in the articles we have a lot of duplicate content. Which is sort of nonsense, since each article is unique. Ah, some might have some common paragraphs because they are recurring news about a weekly competition. But, an example: http://www.betxpert.com/artikler/bookmakere/brandvarme-ailton-snupper-topscorerprisen AND http://www.betxpert.com/artikler/bookmakere/opdaterede-odds-pa-sportschef-situationen-pa-vestegnen These are "duplicate content", however the two article texts are not the same. The menu, and the widgets are all the same, but highly relevant to the article. So what should I do? How can i rid myself of these errors? -Rasmus
On-Page Optimization | | rasmusbang0 -
No Data Available for this URL
Hi,
On-Page Optimization | | ostiguyj
I really don't understand why I have this message "No data available for this URL"
in my SEOMOZ campain. (www.bienchezsoi.ca) When I look at my page rank, I get a score of 0 I have no idea of to fix it. Please help. Thanks0 -
How woud you deal with Blog TAGS & CATEGORY listings that are marked a 'duplicate content' in SEOmoz campaign reports?
We're seeing "Duplicate Content" warnings / errors in some of our clients' sites for blog / event calendar tags and category listings. For example the link to http://www.aavawhistlerhotel.com/news/?category=1098 provides all event listings tagged to the category "Whistler Events". The Meta Title and Meta Description for the "Whistler Events" category is the same as another other category listing. We use Umbraco, a .NET CMS, and we're working on adding some custom programming within Umbraco to develop a unique Meta Title and Meta Description for each page using the tag and/or category and post date in each Meta field to make it more "unique". But my question is .... in the REAL WORLD will taking the time to create this programming really positively impact our overall site performance? I understand that while Google, BING, etc are constantly tweaking their algorithms as of now having duplicate content primarily means that this content won't get indexed and there won't be any really 'fatal' penalties for having this content on our site. If we don't find a way to generate unique Meta Titles and Meta Descriptions we could 'no-follow' these links (for tag and category pages) or just not use these within our blogs. I am confused about this. Any insight others have about this and recommendations on what action you would take is greatly appreciated.
On-Page Optimization | | RoyMcClean0