Is SeoMOZ Crawl Diagnostics wrong here?
-
We've been getting a ton of critical errors (about 80,000) in SeoMoz' Crawl Diagnostics saying we have duplicate content in our client's E-commerce site. Some of the errors are correct, but a lot of the pages are variations like:
www.example.com/productlist?page=1
www.example.com/productlist?page=2
However, in our source code we have used rel="prev" and rel="next" so in my opinion we should be alright.
Would love to hear from you if we have made a mistake or if it is an error in SeoMoz.
Here's a full paste of the script:
-
Just a minor clarification - you can use both rel=prev/next and rel=canonical, IF you have something like search filters. Then, the canonical would point to the unfiltered current page and the rel=prev/next would point to the filtered paginated pages. Yeah, I know, that made a lot of sense. Let's say your page is:
http://example.com/stuff?page=2&sort=price
...then you might have
It's more than a little confusing.
Definitely check out that JavaScript issue, though - it might be that bots aren't seeing what people are seeing, and that could be very dangerous.
-
Hi,
In regards the rel=next you are absolutely right, I must have overlooked it or just searched for the prev tag. So yes as far as proper implementation of the prev/next in that respect it is correct and please ignore that last part of my first post!
Turning of javascript is instructive to see all those tags on their individual page and helps clarify what exactly is being outputted and when without the dynamic loading, providing you don't miss a rel=next tag that is really there
-
Hi Lynn,
Thank you very much for your answer / analysis! As you said "It is a bit confusing" and I will just read your answer a couple of times...
I will grant your answer "Good answer" for you thorough analysis! I think it is spot on with the double "next/prev" and "rel=can" tags. I do have one remark. You said: When I turn off javescript, I get this:
In my opinion this is alright, because it shouldn't have a "prev" as this is the initial page.
-
Hi,
I had a look at what I assume is the site and I think you have a combination of things going on that is likely causing confusion (to you, to the moz bot, probably to google too!)
Firstly, it is not recommended to use rel prev/next and rel canonical on the same page. With that what you are effectively doing is only indexing the first page of the results since all the other pages rel canonical back to the first one. If you have a 'view all' type page then you could rel canonical all of the paginated pages back to this one and you would not need to use the prev/next tags at all. It is also possible that your use of relative canonical links in combination with the above is also causing confusion, usually best to use absolute urls if possible.
Beyond that, the site dynamically loads more products as you scroll down the page which also results in the url changing to hoeretelefon/? for ALL the pages. If that is a problem or not depends on how it is coded and how the google and seomoz bots are deciding to parse the page, but it certainly adds another potential area of complexity to the issue.
Lastly, if you browse the site with javascript turned off you can see something odd in that the initial page /elektronik/baerbar-lyd/hoeretelefon has no prev/next OR canonical tag but has a link to /elektronik/baerbar-lyd/hoeretelefon?page=1 on which you find prev/next and canonicals back to the non paginated version. So you are basically skipping the pagination setup that goes from the original to the page=1 (but also giving a canonical back to the original page).
Phew! It is a bit confusing. I would recommend deciding on if you want to go with prev/next or canonical in the first place and take it from there. I would think that if you have the ability to canonical to a 'see all products page' then this might be the best way to go since it should theoretically take care of any issues the dynamic loading is causing also.
Hope that helps!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Crawl diagnostics incorrectly reporting duplicate page titles
Hi guys, I have a question in regards to the duplicate page titles being reported in my crawl diagnostics. It appears that the URL parameter "?ctm" is causing the crawler to think that duplicate pages exist. In GWT, we've specified to use the representative URL when that parameter is used. It appears to be working, since when I search site:http://www.causes.com/about?ctm=home, I am served a single search result for www.causes.com/about. That begs the question, why is the SEOMoz crawler saying there is duplicate page titles when Google isn't (doesn't appear under the HTML improvements for duplicate page titles)? A canonical URL is not used for this page so I'm assuming that may be one reason why. The only other thing I can think of is that Google's crawler is simply "smarter" than the Moz crawler (no offense, you guys put out an awesome product!). Any help is greatly appreciated and I'm looking forward to being an active participant in the Q&A community! Cheers, Brad
Moz Pro | | brad_dubs0 -
Pages Crawled: 1 Why?
I have some campaigns which have only 1 page crawled, while some other campaigns, having completely similar URL (subdomain) and number of keywords and pages, have all pages crawled... Why is that so? It has been also a while I waited and so far no change...
Moz Pro | | BritishCouncil0 -
SEOmoz giving duplicate content that does not exist.
My problem is similar, and SEOmoz add campaign is giving me several pag. Duplicate, and he's giving me links pag. That do not exist. Look below. My site has 115 pages and the extent SEMOZ gave me 250. Duplicate Page Content ... pages / Alexandra / Clarisse / Clarisse.html
Moz Pro | | Slash-RJ
... pages / Alexandra / Clarisse / Clarisse / Clarisse.html
... pages / Alexandra / Clarisse / Clarisse / Clarisse / Clarisse.html
.... pages / Alexandra / Clarisse / Clarisse / Clarisse / Lizie / Lizie.html When the verade this link does not exist, there is only. ... pages / Alexandra / Alexandra.html
... pages / Clarisse / Clarissehtml
And so on. How to Solve?0 -
Campaigns - crawled
The new Pages Crawled: 2. I have many 404 and other errors, I wanted to start working on it tomorrow but the new crawl only crawled to pages and doesn't show any errors. Whats the problem and what can I do? Yoseph
Moz Pro | | Joseph-Green-SEO0 -
Pages Crawled: 0 ?
I've been with SEO Moz for over a month and a half. Why would this weeks crawl have Pages Crawled: 0? I've made no changes since the crawl last week that had 10k pages crawled...
Moz Pro | | mr_w1 -
How to remove Duplicate content due to url parameters from SEOMoz Crawl Diagnostics
Hello all I'm currently getting back over 8000 crawl errors for duplicate content pages . Its a joomla site with virtuemart and 95% of the errors are for parameters in the url that the customer can use to filter products. Google is handling them fine under webmaster tools parameters but its pretty hard to find the other duplicate content issues in SEOMoz with all of these in the way. All of the problem parameters start with ?product_type_ Should i try and use the robot.txt to stop them from being crawled and if so what would be the best way to include them in the robot.txt Any help greatly appreciated.
Moz Pro | | dfeg0 -
Duplicate Content and Titles in SEOMoz reports
I've had to rename some of the pages on my site and also move them to different locations. I placed a rel="canonical" on the old page pointing to the new one. The reports on my PRO Dashboard are telling me that I have Duplicate Content and Page Title errors. Do the SEOMoz automated reports take the rel="canonical" link into consideration or do I need to remove these pages and do a 301 redirect from the old to the new page?
Moz Pro | | TRICORSystems0 -
My crawl diagnostic is showing 2 duplicate content and titles.
First of all Hi - My name is Jason and I've just joined - How you all doing? My 1st question then: When I view where these errors are occurring it says www mydomain co uk and www mydomain co uk/index.html Isn't this the same page? I have looked into my root folder and only index.html exists.
Moz Pro | | JasonHegarty0