Duplicate pages, overly dynamic URL’s and long URL’s in Magento
-
Hi there,
I’ve just completed the first crawl of my Magento site and SEOMOZ has picked up 1,000’s of duplicate pages, overly dynamic URL’s and long URL’s due to the sort function which appends URL’s with variables when sorting products (e.g. www.example.com?dir=asc&order=duration).
I’m not particularly concerned that this will affect our rankings as Google has stated that they are familiar with the structure of popular CMS’s and Magento is pretty popular.
However it completely dominates my crawl diagnostics so I can’t see if there are any real underlying issues.
Does anyone know a way of preventing this?
Cheers,
Al. -
You should use the Yoast Robots extension to fix almost all the duplicate content.
http://www.magentocommerce.com/magento-connect/yoast-metarobots.html
When using 2.0 Magento connect: http://connect20.magentocommerce.com/community/Yoast_MetaRobots
for 1.0 use: magento-community/Yoast_MetaRobots
Also use canonical URL. You can find this at the admin panel:
System - Configuration - Catalog - Canonical links for catagories
System - Configuration - Catalog - Canonical links for products
-
I'm actually a fan of selectively (programmatically) NOINDEX'ing like that. I find that the GWT parameter blocking doesn't always scale well. I'm running into a lot of clients trying to use it on 100s or 1000s (or millions, actually) of pages and Google is mostly ignoring it. Very frustrating.
We're working on features to let you ignore certain warnings/notices if you feel they don't apply, I but I do believe in being proactive about indexation issues. I think they matter a lot more than they used to, especially post-Panda.
I would double-check to see if there's a Magento plug-in to help, as this could be a common problem. Unfortunately, we don't have any Magento experts on-staff. I'll leave this open as a discussion question, in case any members have specific expertise.
-
Is it worth trying to tackle this programmatically e.g. if url includes dir= or limit= or order= then include a noindex meta tag on that page?
It’s easy to exclude these parameters in Google Webmaster tools, but again I’d really like to reduce the number of errors reported by seoMOZ as currently I have 10,000 errors due to duplicate content!
-
Hey Harald, Thanks for your response - I've come across that article whilst googling the issue, but it doesn't specifically deal with the duplicate URL's being crawled and being included in SEOmoz reports. As I say I'm not too worried about any negative impact here as I've implemented canonical URL's and I have a sitemap - however it ruins my SEOmoz crawl diagnostic report by creating 1,000's of errors. Cheers, Al.
-
Hi Almenzies, As you mentioned that SEOmoz repots you by telling that there area 1000 of pages which are having the issues of duplicate content , so below is alink which solves the Duplicate content issues:
Solving the Duplicate Content Issues in Magento.
I hope that your query had been solved.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Duplicate Page Titles For Paginated Topics In Blog
Hello, I've just run a site audit and it has come up with a duplicate title tag issue for the topics section of our blog. For example it is flagging that the following have the same page title. https://blog.companyname.com/topic/topic-name https://blog.companyname.com/topic/topic-name/page/2 How significant is this as an SEO issue and what are the ways we can go about fixing this? I look forward to any suggestions and guidance that can be provided. Thanks, John
Technical SEO | | SEOCT1 -
My website's pages are not being indexed correctly
Hi, One of our websites, which is actually a price comparison engine, facing indexing problem at Google. When we check “site:mywebsite.com “, there are lots of pages indexed which are not from mywebsite.com but from merchants websites. The index result page also shows merchant’s page title. In some cases the title is from merchant’s site but when the given link is accessed it points to mywebsite.com/index. Also the cache displays the merchant’s product page as the last indexed version rather than showing ours. The mywebsite.com has quite few Merchants that send us their product feed. Those products are listed on comparison page with prices. The merchant’s links on comparison page are all no-follow links but some of the (not all) merchant’s product pages are indexed against mywebsite.com as mentioned above instead of product comparison page of mywebsite.com How can we fix the issue? Thanks!
Technical SEO | | digitalMSB0 -
How long after disallowing Googlebot from crawling a domain until those pages drop out of their index?
We recently had Google crawl a version of the site we that we had thought we had disallowed already. We have corrected the issue of them crawling the site, but pages from that version are still appearing in the search results (the version we want them to not index and serve up is our .us domain which should have been blocked to them). My question is this: How long should I expect that domain (the .us we don't want to appear) to stay in their index after disallowing their bot? Is this a matter of days, weeks, or months?
Technical SEO | | TLM0 -
Duplicate title error in GWT over spelling in URL
Hi, How do I resolve a duplicate title error in GWT over spelling in URL? Ttile of Post: Minneapolis Median Home Sales Price Up 16 Percent Not sure how this happened, but I have two URL versions show up. Even with a 301 redirect, the both remain an error in GWT. /real-estate-blog/Minneapolis-median-home-sales-price-up-16-percent and /real-estate-blog/minneapolis-median-home-sales-price-up-16-percent
Technical SEO | | jessential0 -
Keyword Targeting with Dynamic Pages
We have a large e-commerce website made with .net. so all of our category and item pages are made dynamic. Most things like title, some of the words and a few other things are done with scripts. I want to be able to target certain words and have more customized words on certain pages. Has anyone dealt with this? I know .net is pretty common so I can't be a unique case.
Technical SEO | | EcommerceSite0 -
Are you allowed to point different urls to same page
hi, i have some urls that i am either going to put on hold or thinking about pointing to one of my sites. what it is, i am looking at re-designing the pages but not until next year, so i thought i would point some of the urls to a site that i am happy with to different pages, but not sure if i am allowed this or not so for example, if i have a site on cars, and one of the url is www.rovercars.co.uk i was thinking about pointing it to the page that is about rover cars. can anyone let me know if this is allowed or not please
Technical SEO | | ClaireH-1848860 -
GWT, URL Parameters, and Magento
I'm getting into the URL parameters in Google Webmaster Tools and I was just wondering if anyone that uses Magento has used this functionality to make sure filter pages aren't being indexed. Basically, I know what the different parameters (manufacturer, price, etc.) are doing to the content - narrowing. I was just wondering what you choose after you tell Google what the parameter's function is. For narrowing, it gives the following options: Which URLs with this parameter should Googlebot crawl? <label for="cup-crawl-LET_GOOGLEBOT_DECIDE">Let Googlebot decide</label> (Default) <label for="cup-crawl-EVERY_URL">Every URL</label> (the page content changes for each value) <label style="color: #5e5e5e;" for="cup-crawl-ONLY_URLS_WITH_VALUE">Only URLs with value</label> ▼(may hide content from Googlebot) <label for="cup-crawl-NO_URLS">No URLs</label> I'm not sure which one I want. Something tells me probably "No URLs", as this content isn't something a user will see unless they filter the results (and, therefore, should not come through on a search to this page). However, the page content does change for each value.I want to make sure I don't exclude the wrong thing and end up with a bunch of pages disappearing from Google.Any help with this is greatly appreciated!
Technical SEO | | Marketing.SCG0 -
Rel cannonical on all my URL's
Hi, sorry if this question has already been asked, but I can't seem to find the correct answer. In my crawling report for the domain: http://www.wellbo.de I get rel cannonical notices. I have redirected all pages of http://wellbo.de to http://www.wellbo.de with a 301 redirect. Where is my error? Why do I get these notices? I hope the image helps. Ep7Rw.jpg
Technical SEO | | wellbo0