Duplicate pages coming from links from the login page - what should we do about them?
-
This is a follow on to an earlier question which was well answered by Dirk Ceuppens regarding abnormal crawl issues. We are seeing that the issues relating to Duplicate Pages are coming from links from the login page which shows information about where the user was redirected from.
For example, if the visitor is not logged on and wishes to wish-list an item, they will be redirected to the login page, with the item code and intended action in the url; which can then continue on to the desired page once logged on.
The MOZ crawler is seeing these pages as having Duplicated Content whilst they are all the same apart from a piece of information in the URL. Should we be blocking these duplications? Are they a risk to us? What should we be doing?
Many thanks,
Sarah
-
Hi Sarah,
Somehow I answered this and I must have forgotten to post the answer! Arg, it was a long one, too. Let me try to summarize what I'd do:
-If possible, noindex any page that doesn't display content while not logged in. Wait for those pages to drop out of the index, and monitor for errors.
- If not possible, skip straight to blocking pages behind a login wall with robots.txt. For example, to block anything in the login folder:
Disallow: /login
Or to block anything with a login variable:
Disallow: /*?login
This should prevent bots from crawling those URLs where you don't have any content to show them. Make sure to use this carefully.
I do apologize for the delay. If you have additional questions please feel free to PM me. I'd be happy to do a quick consult online or over the phone, as I feel bad that I never actually answered, and I can give you more specific ideas if we look at the site. If this answers your question that's fine too.
Good luck!
-
Hi Sarah,
I missed this notification on this one somehow!
To be honest, I don't have an answer for you on this one. Perhaps it might be worth either getting in touch with the Moz team or posting another question specifically tagged as "Product Support". They seem to be pretty good at answering those queries too
-
Thanks for this Chris.
One other thing, how then do I block this from showing up in my MOZ crawl, which is giving me 16,9k crawl issues and also how do i then work out what the other crawl issues are that are mixed up in this huge report?
-
Honestly I wouldn't be real worried about it. It seems Google is smart enough these days to understand what's going on there though canonicalization would be wise - just point the canonical tag on the login page to itself.
By doing this, assuming your URLs look something like domain.com/login?product=-product-name, all variations will theoretically be seen as the /login page.
If you really wanted to, you could use Robots to block these as well but I honestly wouldn't bother.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Crawl Diagnostics saids a page is linking but I can't find the link on the page.
Hi I have just got my first Crawl Diagnostics report and I have a questions. It saids that this page: http://goo.gl/8py9wj links to http://goo.gl/Uc7qKq which is a 404. I can't recognize the URL on the page which is a 404 and when searching in the code I can't find the %7Blink%7D in the URL which gives the problems. I hope you can help me to understand what triggers it 🙂
Moz Pro | | SebastianThode0 -
Moz crawl duplicate pages issues
Hi According to the moz crawl on my website I have in the region of 800 pages which are considered internal duplicates. I'm a little puzzled by this, even more so as some of the pages it lists as being duplicate of another are not. For example, the moz crawler considers page B to be a duplicate of page A in the urls below: Not sure on the live link policy so ive put a space in the urls to 'unlive' them. Page A http:// nuchic.co.uk/index.php/jeans/straight-jeans.html?manufacturer=3751 Page B http:// nuchic.co.uk/index.php/catalog/category/view/s/accessories/id/92/?cat=97&manufacturer=3603 One is a filter page for Curvety Jeans and the other a filter page for Charles Clinkard Accessories. The page titles are different, the page content is different so Ive no idea why these would be considered duplicate. Thin maybe, but not duplicate. Like wise, pages B and C are considered a duplicate of page A in the following Page A http:// nuchic.co.uk/index.php/bags.html?dir=desc&manufacturer=4050&order=price Page B http:// nuchic.co.uk/index.php/catalog/category/view/s/purses/id/98/?manufacturer=4001 Page C http:// nuchic.co.uk/index.php/coats/waistcoats.html?manufacturer=4053 Again, these are product filter pages which the crawler would have found using the site filtering system, but, again, I cannot find what makes pages B and C a duplicate of A. Page A is a filtered result for Great Plains Bags (filtered from the general bags collection). Page B is the filtered results for Chic Look Purses from the Purses section and Page C is the filtered results for Apricot Waistcoats from the Waistcoat section. I'm keen to fix the duplicate content errors on the site before it goes properly live at the end of this month - that's why anyone kind enough to check the links will see a few design issues with the site - however in order to fix the problem I first need to work out what it is and I can't in this case. Can anyone else see how these pages could be considered a duplicate of each other please? Checking ive not gone mad!! Thanks, Carl
Moz Pro | | daedriccarl0 -
Joomla Opencart Duplicate Link Issue
With a few recent catalog changes my client is having tons of duplicate content issues. The most peculiar is one that I cannot see visibly on the site pages. This page: http://www.CLIENTDOMAIN.com/catalog/lights.html shows to have several duplicate content issues with URLs such as: http://www.CLIENTDOMAIN.com/catalog/lights/Lamp-DRW.htmlhttp:/www.CLIENTDOMAIN.com - where the domain is basically added to the end of the first URL. Has anyone seen this? Any fixes, plugins or thoughts? Thanks,
Moz Pro | | KristinaWitmer
Kristina0 -
NoFollow Links from Subdomain to root domain better than DoFollow Links?
Our service at fotograf.de is a shopsystem for professional photographers. The customers can build their own website with our tool including an onlineshop to sell their pictures. Here is my question: One part of the customers use subdomains of our site like photographers.fotograf.de. On each customer website we include a backlink to our homepage www.fotograf.de. From SEO view is it better to set these links as NoFollow Links? Or should we put one Follow Link on the starting page on each site and on the other pages only NoFollow Link? Are these links bad for our SEO regarding link diversity because they all come from one root domain? Thanks for the answers! Sebastian
Moz Pro | | Sebastian230 -
How Old is OSE link data?
I ran an anchor text report for my client today, which shows that their site has some incoming comment spam links using totally unrelated phrases (pharma products). However, when looking for the live link, the linking page no longer contains the link to them. Maybe the webmasters removed these, but I can't track down a single one... how old is this data? thanks
Moz Pro | | JMagary0 -
About Duplicate Content found by SEOMOZ... that is not duplicate
Hi folks, I am hunting for duplicate content based on SEOMOZ great tool for that 🙂 I have some pages that are mentioned as duplicate but I cant say why. They are video page. The content is minimalistic so I guess it might be because all the navigation is the same but for instance http://www.nuxeo.com/en/resource-center/Videos/Nuxeo-World-2010/Nuxeo-World-2010-Presentation-Thierry-Delprat-CTO and http://www.nuxeo.com/en/resource-center/Videos/Nuxeo-World-2010/Nuxeo-World-2010-Presentation-Cheryl-McKinnon-CMO are mentioned as duplicate. Any idea? Is it hurting? Cheers,
Moz Pro | | nuxeo0 -
Only Crawling 1 page?
Hi Guys, Any advice much appreciated on this! Recently set up a new campaign on my dashboard with just 5 keywords. The domain is brammer.co.uk and a quick Google site:brammer.co.uk shows a good amount of indexed pages. However - first seomoz tool crawl has only crawled 1 url!! "Last Crawl Completed: Apr. 12th, 2011 Next Crawl Starts: Apr. 17th, 2011" Any ideas what's stopping the tool crawl anymore of the site?? Cheers in advance.. J
Moz Pro | | lovealbatross0 -
What do i do when all pages are grade A?
I've used the on page grade and now have all my pages at a grade A for relevant keywords. Most of them are cool, achieveing first page rankings apart from a few massive keywords. So the question is, what's next? What do i do now that I'm at grade A, but perhaps not #1 yet... Cheers -dan
Moz Pro | | spytunes0