Duplicate blog URLs in Magenton
-
On one my sites Moz is picking up 4483 duplicate content pages. The majority of these are from our blog and video sections on our site.
We're using a URL shortener and it appears that some of the pages are the full version of the URL then the shortened version. However if you go to the full version you get redirected to the shorter one. So I would assume that the Moz crawler should get the same redirect?
We're also getting pagination being shown as duplicate pages, which I would half expect, but the URLs Magento is creating are truly bizarre:
e.g http://www.xxx.com/uk/blog/cat/view/identifier/news/page/news/index.php/alarms-doorbells/?p=2
Alarms and doorbells is one of our product categories, which is displayed in the LHN on the blog page but has nothing to do with the blog itself.
On another site on the same Magento instance, with the same content (they're for two different regions) we're show as having 248 duplicate pages, again in the video and news section, but this is a completely different scale of issue.
Has anyone else encountered issues like these? I'm probably going to put a noindex in place on these two sections until we can get a solution in place as we're completely unranked in google on this site.
Thanks
-
I actually had a look at the robots.txt last night to block the effected areas and discovered the dev's who built the site hadn't set it up. We've added a default one which we're going to test over the next week to see how it fares. If that doesn't work then we'll start looking at the parameters.
Thanks.
-
Hi,
Have you tried setting url parameters in webmaster tools? This can help you to tell Google which urls not to crawl depending on certain parameters, such as if the url has "?p"
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Duplicate Page Titles For Paginated Topics In Blog
Hello, I've just run a site audit and it has come up with a duplicate title tag issue for the topics section of our blog. For example it is flagging that the following have the same page title. https://blog.companyname.com/topic/topic-name https://blog.companyname.com/topic/topic-name/page/2 How significant is this as an SEO issue and what are the ways we can go about fixing this? I look forward to any suggestions and guidance that can be provided. Thanks, John
Technical SEO | | SEOCT1 -
Duplicate Content
I am trying to get a handle on how to fix and control a large amount of duplicate content I keep getting on my Moz Reports. The main area where this comes up is for duplicate page content and duplicate title tags ... thousands of them. I partially understand the source of the problem. My site mixes free content with content that requires a login. I think if I were to change my crawl settings to eliminate the login and index the paid content it would lower the quantity of duplicate pages and help me identify the true duplicate pages because a large number of duplicates occur at the site login. Unfortunately, it's not simple in my case because last year I encountered a problem when migrating my archives into a new CMS. The app in the CMS that migrated the data caused a large amount of data truncation Which means that I am piecing together my archives of approximately 5,000 articles. It also means that much of the piecing together process requires me to keep the former app that manages the articles to find where certain articles were truncated and to copy the text that followed the truncation and complete the articles. So far, I have restored about half of the archives which is time-consuming tedious work. My question is if anyone knows a more efficient way of identifying and editing duplicate pages and title tags?
Technical SEO | | Prop650 -
Duplicate Product Descriptions
Hi All, After a bit of advice please, we will be selling similar products e.g. A chair which comes in different fabrics and finishes, but is basically the same product. Most, if not all of the ‘long’ product descriptions are identical with only the ‘short’ product descriptions being unique. The ‘long’ product descriptions contain all the manufacturing information, leg option/colour information, graphics, dimensions, weight etc etc. I’m concerned that by having 300+ products all with identical ‘long’ descriptions its going to be seen negatively by google and effect the sites SEO. My question is will this be viewed as duplicate content? If so, are there any best practices I should be following for handling this, other than writing completely unique descriptions for each product, which would be extremely difficult given its basically the same products re-hashed. Many thanks in advance for any advice.
Technical SEO | | Jon-S0 -
Duplicate Content Question
I have a client that operates a local service-based business. They are thinking of expanding that business to another geographic area (a drive several hours away in an affluent summer vacation area). The name of the existing business contains the name of the city, so it would not be well-suited to market 'City X' business in 'City Y'. My initial thought was to (for the most part) 'duplicate' the existing site onto a new site (brand new root domain). Much of the content would be the exact same. We could re-word some things so there aren't entire lengthy paragraphs of identical info, but it seems pointless to completely reinvent the wheel. We'll get as creative as possible, but certain things just wouldn't change. This seems like the most pragmatic thing to do given their goals, but I'm worried about duplicate content. It doesn't feel as though this is spammy though, so I'm not sure if there's cause for concern.
Technical SEO | | stevefidelity0 -
Duplicate Content
Hello guys, After fixing the rel tag on similar pages on the site I thought that duplicate content issue were resolved. I checked HTML Improvements on GWT and instead of going down as I expected, it went up. The duplicate issues affect identical product pages which differ from each other just for one detail, let's say length or colour. I could write different meta tags as the duplicate is the meta description, and I did it for some products but still didn't have any effects and they are still showing as duplicates. What would the problem be? Cheers
Technical SEO | | PremioOscar0 -
URLs Case Sensitive Serving Duplications
At PakWheels.com we have URLs being generated in upper and lower cases. For example following URLs serve pages: http://www.pakwheels.com/used-cars/search/-/mk_Toyota/ http://www.pakwheels.com/used-cars/search/-/mk_toyota/ Both show same content. Similarly in following four cases: http://www.pakwheels.com/used-cars/search/-/mk_Toyota/md_corolla/ http://www.pakwheels.com/used-cars/search/-/mk_Toyota/md_Corolla/ http://www.pakwheels.com/used-cars/search/-/mk_toyota/md_Corolla/ http://www.pakwheels.com/used-cars/search/-/mk_toyota/md_corolla/ all of these 4 URLs serve page with same content. What is the best practice for this issue, is it generating duplication? Please advise
Technical SEO | | razasaeed0 -
I'm getting duplicate content created with a random string of character added to the end of my blog post permalinks?
In an effort to clean up my blog content I noticed that I have a lot of posts getting tagged for duplicate content. It looks like ... http://carwoo.com/blog/october-sales-robust-stateside-european-outlook-poor-for-ford http://carwoo.com/blog/october-sales-robust-stateside-european-outlook-poor-for-ford/954bf0df0a0d02b700a06816f2276fa5/ Any thoughts on how and why this would be happening?
Technical SEO | | editabletext0 -
Removing duplicate &var=1 etc var name urls from google
Hi I had a huge drop in traffic around the 11th of july over 50% down with no recovery as yet... ~5000 organic visits per day down to barley over 2500. I fixed up a problem that one script was introducing that had caused high bounce rates. Now i have identified that google has indexed the entire news section 4 times, same content but with var=0 var=1 2 3 etc around 40,000 urls in total. Now this would have to be causing problems. I have fixed the problem and those url's 404 now, no need for 301's as they are not linked to from anywhere. How can I get them out of the index? I cant do it one by one with the url removal request.. I cant remove a directory from url removal tool as the reuglar content is still there.. If I ban it in robots.txt those urls, wont it never try to index them again and thus not ever discover they are 404ing? These urls are no longer linked to from anywhere, so how can google ever reach them by crawling to find them 404ing?
Technical SEO | | Adsau0