How to remove hundreds of duplicate pages

[Modi]

Hi - while i was checking duplicate links, am finding hundreds of duplicates pages :-

• having undefined after domain name and before sub page url

• having /%5C%22/ after domain name and before the sub page url

• Due to Pagination limits

Its a joomla site - http://www.mycarhelpline.com

Any suggestions - shall we use:-

• 301 redirect

• leave these as standdstill

• and what to do of pagination pages (shall we create a separate title tag n meta description of every pagination page as unique one)

thanks

[RebekahMay]

Okay, I took a look at the plugin Ben recommended, and took another look at the SH404SEF one.  The free one Ben recommended (http://extensions.joomla.org/extensions/site-management/sef/1063) looks like it can help out with some duplicate content - but what I recommend doing is getting the SH404SEF here http://anything-digital.com/sh404sef/features.html because it allows for setting up canonical tags and also gives you the option to add the rel=next feature to your paginated pages, which is one of your problem areas.

One thing I noticed though is that it specifically states it "automatically adds canonical tags to non-html pages" - so that means it will apply it automatically to Joomla's defaul pdf view, etc.  While this is helpful, it may not solve the full issue of your duplicate pages with the undefined and "/%5C%22/" issue.

It does however state that it "removes duplicate URLs" - how it identifies and removes these, I am not sure.  You may want to try it out because it is useful for other optimization tasks - or contact the owner for more information.

If the tool doesn't recognize and remove the duplicate pages caused by /undefined/ and "/%5C%22/" then you should disallow crawling of these in your robots.txt file.  While you are in your robots.txt file you should remove the /images/ because you want those to be crawled - Joomla adds that in by default.

Because a lot of these pages have already been crawled, you should do a 301 on the duplicate pages to their matching page.  This sounds like it will be a long process - this may be aided by the sh404sef plugin - not sure.

I just want to also add that I am in no way affiliated with any of these plugins.

[BenRWoodard]

The only way to solve the duplication error you are getting is to make the URL's distinct.  Googlebot comes to your site and looks at the URL's and if they are not distinct it may not index them very well.  I understand your site is showing up fine in the SERP's so this may be one of those items you place on a lower priority until later.

I think R.May knows Joomla so I'll refer to him on how to accomplish this but it may be worth it to make the adjustment.  You may find the end result of making your page URL more distinct will actually increase your current SERPs.  Just a thought.

Other than that. If your site isn't hurting and the only thing you are concerned about is the report in SEOmoz then I would move on and just make a mental note of it for later.

[Modi]

Hi ben - changing url is not well required as the site is getting good serp, however - the duplicacy issue to saveguard us from any future issue - is what we seek for

[Modi]

Hi - thanks for replying

• For the dynamic url - Yes - at the initial start - it was missed and as on its not reqd somehow - as the pages are getting indexed well and good in SERP

• For pagination - Where we needs this is like in our used car section, discount section& news section where multiple pages are created. shall we create separate title & meta description for every pagination page. is it ideally reqd ?

http://www.mycarhelpline.com/index.php?option=com_usedcar&view=category&Itemid=3

- 'undefined' &  /%5C%22/ is coming as per report of SEOmoz and is almost on every page of site (except of home page) with the dynamic url after domain name are preceded with these 2 strings as per moz report

how to get this corrected - want to be preventive from this duplicacy n avoid getting a hit in future even if its going well now -

[BenRWoodard]

I'm not a Joomla expert but to make your URL's search engine friendly you are going to need to add an extension like this.  That will allow you to make more distinct URLs that will not be considered "duplicate" anymore.

[RebekahMay]

Joomla has soo many dup content issues, you have to know Joomla really well to avoid most of them.  The biggest issue is you didn't enable the SEF URLs from the start and left the default index.php?option=com on most of them, which stuffs your URLs full of ugly parameters.

You can still enable this in your global options and with a quick edit to htaccess  - but it will change all of your current URLs and you will need to 301 all of them, so that isn't a great option unless you are really suffering - and depending on if you are using J 1.6 or under, this is a time consuming nasty process.  Also this is unlikely to get rid of any existing duplicate pages - but may make dealing with them and finding them easier.

I don't see the specific examples you posted though, where are you seeing "undefined" and "%5C%22/ " ?

You should implement rel=canonical on the correct version of each page.  I recommend SH404SEF which is a Joomla plugin and makes this process easier - but it isn't free.  I don't know of a good free plugin that does this, and Joomla's templates make doing this manually difficult.

Looking at it quickly, I also didn't notice any articles that were paginated, but you should try to follow the rel="next" and rel="prev" for paginated pages.  This is likely something you will have to edit your Joomla core files to do.