Penalization for Duplicate URLs with %29 or "/"
-
Hi there -
Some of our dynamically generated product URLs somehow are showing up in SEOmoz as two different URLs even though they are the same page- one with a %28 and one with a
e.g.,
http://www.company.com/ProductX-(-etc/
http://www.company.com/ProductX-(-etc/
Also, some of the URLs are duplicated with a "/" at the end of them.
Does Google penalize us for these duplicate URLs? Should we add canonical tags to all of them?
Finally, our development team is claiming that they are not generating these pages, and that they are being generated from facebook/pinterest/etc. which doesn't make a whole lot of sense to me. Is that right?
Thanks!
-
Canonical tags should drastically help with this. The % is being generated because the URL is being encoded and has a "(" in it. Have your product page each contain their own canonical with the URL you want indexed. Not sure which URL to use? Check your internal links and see how your site is linking to your product pages. Presumably its:
http://www.company.com/ProductX-(-etc/
or
http://www.company.com/ProductX-(-etc
Add this URL as your canonical and the SE's will understand what page is the 'real' page. This will solve both problems from an SEO standpoint. If you want to actually stop the site from doing this, you can remove trailing slashes and encoding using HTACCESS.
-
In short, your site software should completely control all links generated on your site. If you hand code a site using .NET, Cold Fusion, HTML/CSS/PHP, etc. you are in complete control over your links. If you use a CMS or other software such as WordPress, Magento, etc. then the software creates the URLs for you. In either case a skilled developer should be able to offer you options.
In brief, I recommend using a standard format for your URLs. We like to have all categories end with a trailing slash, and all web pages end without one. For example: www.mysite.com/cars/ or www.mysite.com/cars/2010-ford-mustang.
Whatever choice you make, enforce it throughout your site. You can also use the canonical tag to help control issues where a page may be offered under multiple URLs, but the best choice would be solving the root issue.
**Our development team is claiming that they are not generating these pages, and that they are being generated from facebook/pinterest/etc. which doesn't make a whole lot of sense to me. **
Without looking at your website and an example URL of this issue, it is not possible to offer a definitive answer. I have never encountered this issue.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Both links with ".html" and without are working , Is that a problem ?
Default format of my url ending with ".html" , I know it's not a problem .. But both links with ".html" and without are working , Is that critical problem or not ? and how to solve it ?
Technical SEO | | Mohamed_Samer0 -
Why are these two URLs showing in Moz as duplicate content?
Here is the first URL - http://www.flagandbanner.com/Products/FBPP0000012376.asp Here is the 2nd URL - http://www.flagandbanner.com/Products/flag-spreader.asp Granted I am new to this issue on this website, but what is Roger seeing that I'm not? A lot of our duplicate pages are just like this example.
Technical SEO | | Flaglady0 -
Robots.txt to disallow /index.php/ path
Hi SEOmoz, I have a problem with my Joomla site (yeah - me too!). I get a large amount of /index.php/ urls despite using a program to handle these issues. The URLs cause indexation errors with google (404). Now, I fixed this issue once before, but the problem persist. So I thought, instead of wasting more time, couldnt I just disallow all paths containing /index.php/ ?. I don't use that extension, but would it cause me any problems from an SEO perspective? How do I disallow all index.php's? Is it a simple: Disallow: /index.php/
Technical SEO | | Mikkehl0 -
How unique does a page need to be to avoid "duplicate content" issues?
We sell products that can be very similar to one another. Product Example: Power Drill A and Power Drill A1 With these two hypothetical products, the only real difference from the two pages would be a slight change in the URL and a slight modification in the H1/Title tag. Are these 2 slight modifications significant enough to avoid a "duplicate content" flagging? Please advise, and thanks in advance!
Technical SEO | | WhiteCap0 -
Duplicate Page Warnings, hkey and repetitive URLs
Hi, we just put our association's site in SEO Moz so we can tackle SEO and we received thousands of duplicate page content and duplicate title warnings. I searched the forum before asking 🙂 Appreciate some guidance on how many things are wrong with the URL's below that are getting flagged as duplicate page content. 1. Does the repetition of the page title and section hurt SEO? 2. Does the iMIS15 in the URL (the server) detract from relevant ranking? 3. From the forum, it looks like canonical tags should be added to the version of the page that starts with .....?hkey (is there a way to predict and "canonize" these? Or recommendations?) http://www.iiba.org/imis15/IIBA/About_IIBA/IIBA_Website/About_IIBA/About_IIBA.aspx?hkey=6d821afa-e3aa-4bef-bafa-2453238d12c6 http://www.iiba.org/imis15/IIBA/About_IIBA/IIBA_Website/About_IIBA/About_IIBA.aspx Thank you.
Technical SEO | | lyndas0 -
How "Optimised" is my home page content
Good afternoon from 1 degrees C overcast frozen wetherby UK... I've made a number of on page html markup changes to optimise the page for steel suppliers steel stockholders but I'd like to know if there are any other on page improvments I could make for this page http://www.barrettsteel.com/ Im particulary concerned that contnet in in li tags and not p, could this be an issue? And finaaly on the home page a third party developer has slapped a header banner pointing to an external site know as woodberry tools, that cant be good can it? Any insights welcome 🙂
Technical SEO | | Nightwing0 -
Multiple URLs in CMS - duplicate content issue?
So about a month ago, we finally ported our site over to a content management system called Umbraco. Overall, it's okay, and certainly better than what we had before (i.e. nothing - just static pages). However, I did discover a problem with the URL management within the system. We had a number of pages that existed as follows: sparkenergy.com/state/name However, they exist now within certain folders, like so: sparkenergy.com/about-us/service-map/name So we had an aliasing system set up whereby you could call the URL basically whatever you want, so that allowed us to retain the old URL structure. However, we have found that the alias does not override, but just adds another option to finding a page. Which means the same pages can open under at least two different URLs, such as http://www.sparkenergy.com/state/texas and http://www.sparkenergy.com/about-us/service-map/texas. I've tried pointing to the aliased URL in other parts of the site with the rel canonical tag, without success. How much of a problem is this with respect to duplicate content? Should we bite the bullet, remove the aliased URLs and do 301s to the new folder structure?
Technical SEO | | ufmedia0 -
I have a site that has both http:// and https:// versions indexed, e.g. https://www.homepage.com/ and http://www.homepage.com/. How do I de-index the https// versions without losing the link juice that is going to the https://homepage.com/ pages?
I can't 301 https// to http:// since there are some form pages that need to be https:// The site has 20,000 + pages so individually 301ing each page would be a nightmare. Any suggestions would be greatly appreciated.
Technical SEO | | fthead90