How to prevent duplicate content within this complex website?
-
I have a complex SEO issue I've been wrestling with and I'd appreciate your views on this very much. I have a sports website and most visitors are looking for the games that are played in the current week (I've studied this - it's true). We're creating a new website from scratch and I want to do this is as best as possible. We want to use the most elegant and best way to do this. We do not want to use work-arounds such as iframes, hiding text using AJAX etc. We need a solid solution for both users and search engines.
Therefor I have written down three options:
- Using a canonical URL;
- Using 301-redirects;
- Using 302-redirects.
Introduction
The page 'website.com/competition/season/week-8' shows the soccer games that are played in game week 8 of the season. The next week users are interested in the games that are played in that week (game week 9). So the content a visitor is interested in, is constantly shifting because of the way competitions and tournaments are organized. After a season the same goes for the season of course.
The website we're building has the following structure:
- Competition (e.g. 'premier league')
- Season (e.g. '2011-2012')
- Playweek (e.g. 'week 8')
- Game (e.g. 'Manchester United - Arsenal')
- Playweek (e.g. 'week 8')
- Season (e.g. '2011-2012')
This is the most logical structure one can think of. This is what users expect.
Now we're facing the following challenge: when a user goes to http://website.com/premier-league he expects to see a) the games that are played in the current week and b) the current standings. When someone goes to http://website.com/premier-league/2011-2012/ he expects to see the same: the games that are played in the current week and the current standings. When someone goes to http://website.com/premier-league/2011-2012/week-8/ he expects to the same: the games that are played in the current week and the current standings.
So essentially there's three places, within every active season within a competition, within the website where logically the same information has to be shown.
To deal with this from a UX and SEO perspective, we have the following options:
Option A - Use a canonical URL
Using a canonical URL could solve this problem. You could use a canonical URL from the current week page and the Season page to the competition page:
So:
- the page on 'website.com/$competition/$season/playweek-8' would have a canonical tag that points to 'website.com/$competition/'
- the page on 'website.com/$competition/$season/' would have a canonical tag that points to 'website.com/$competition/'
The next week however, you want to have the canonical tag on 'website.com/$competition/$season/playweek-9' and the canonical tag from 'website.com/$competition/$season/playweek-8' should be removed.
So then you have:
- the page on 'website.com/$competition/$season/playweek-9' would have a canonical tag that points to 'website.com/$competition/'
- the page on 'website.com/$competition/$season/' would still have a canonical tag that points to 'website.com/$competition/'
In essence the canonical tag is constantly traveling through the pages.
Advantages:
- UX: for a user this is a very neat solution. Wherever a user goes, he sees the information he expects. So that's all good.
- SEO: the search engines get very clear guidelines as to how the website functions and we prevent duplicate content.
Disavantages:
- I have some concerns regarding the weekly changing canonical tag from a SEO perspective. Every week, within every competition the canonical tags are updated. How often do Search Engines update their index for canonical tags? I mean, say it takes a Search Engine a week to visit a page, crawl a page and process a canonical tag correctly, then the Search Engines will be a week behind on figuring out the actual structure of the hierarchy. On top of that: what do the changing canonical URLs to the 'quality' of the website? In theory this should be working all but I have some reservations on this.
- If there is a canonical tag from 'website.com/$competition/$season/week-8', what does this do to the indexation and ranking of it's subpages (the actual match pages)
Option B - Using 301-redirects
Using 301-redirects essentially the user and the Search Engine are treated the same. When the Season page or competition page are requested both are redirected to game week page.
The same applies here as applies for the canonical URL: every week there are changes in the redirects.
So in game week 8:
- the page on 'website.com/$competition/' would have a 301-redirect that points to 'website.com/$competition/$season/week-8'
- the page on 'website.com/$competition/$season' would have a 301-redirect that points to 'website.com/$competition/$season/week-8'
A week goes by, so then you have:
- the page on 'website.com/$competition/' would have a 301-redirect that points to 'website.com/$competition/$season/week-9'
- the page on 'website.com/$competition/$season' would have a 301-redirect that points to 'website.com/$competition/$season/week-9'
Advantages
- There is no loss of link authority.
Disadvantages
- Before a playweek starts the playweek in question can be indexed. However, in the current playweek the playweek page 301-redirects to the competition page. After that week the page's 301-redirect is removed again and it's indexable.
- What do all the (changing) 301-redirects do to the overall quality of the website for Search Engines (and users)?
Option C - Using 302-redirects
Most SEO's will refrain from using 302-redirects. However, 302-redirect can be put to good use: for serving a temporary redirect.
Within my website there's the content that's most important to the users (and therefor search engines) is constantly moving. In most cases after a week a different piece of the website is most interesting for a user. So let's take our example above. We're in playweek 8.
If you want 'website.com/$competition/' to be redirecting to 'website.com/$competition/$season/week-8/' you can use a 302-redirect. Because the redirect is temporary
The next week the 302-redirect on 'website.com/$competition/' will be adjusted. It'll be pointing to 'website.com/$competition/$season/week-9'.
Advantages
- We're putting the 302-redirect to its actual use.
- The pages that 302-redirect (for instance 'website.com/$competition' and 'website.com/$competition/$season') will remain indexed.
Disadvantages
- Not quite sure how Google will handle this, they're not very clear on how they exactly handle a 302-redirect and in which cases a 302-redirect might be useful. In most cases they advise webmasters not to use it.
I'd very much like your opinion on this. Thanks in advance guys and galls!
-
Hi Andy and Peter, thanks for your response.
@Andy: the rel=next and rel=prev markup won't really help in solving the problem we had. We will use it though because it's very helpful.
@Peter: yeah it's been something we've been struggling with for a while but we've finally made a decision on it.
The /current solution wasn't really a good solution because at the start of a season all the gameweeks are planned and created so it would become quite complex. We've done some calculations on how much duplicate content we would have if we would not use any of the redirects of canonical tags and the percentage of DC is very small (below 1%) so we're going to put our faith in Google's hands and let them figure it out. It's a good quality website with loads of links we're talking about so I don't expect to much issues. We'll monitor it closely though and stand by to interfere when needed.
Anyways, thanks for your suggestions. Although it didn't solve my problem 1:1 it did make me think and make a decision.
Bye, Steven
-
Yeah, time-sensitive information is always tough. I think you're dead on about the disadvantages - the timing of Google's application of these rotating tags would always be off, and you could end up with some really weird search results that are not only bad for SEO but could create bad UX (people landing on old pages thinking they're new).
What about another option - could you take more of a news/blog approach and have a "/current" page that is always the current week? As the current week changes, roll that content into an archive page ("/week8", etc.). That way, the content lives on, but the current URL never changes.
In terms of duplication, is this really full duplication? It sounds like some pages (like the season) just have snippets of the current week. That's not necessarily a problem. If they are very similar, could you "widgetize" it somehow? Could be straight HTML, but use a condensed format for the season page that links to the full version on the current week page. This would be much like a snippet of a blog post - instead of repeating everything on all 3 pages, have one main chunk of content and two summaries.
-
Hi,
Does the rel=next, rel=prev markup help you out with this problem? See http://googlewebmastercentral.blogspot.co.uk/2011/09/pagination-with-relnext-and-relprev.html
Ive used it a couple of times to help stop pages been seen as dupe content where those pages are duplicate (meta, main content, images etc) except for example reviews, or comments e.g. /product_x /product_x_review_page1 /product_x_review_page2 /product_x_review_page_3
Andy
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Duplicate Content That Isn't Duplicated
In Moz, I am receiving multiple messages saying that there is duplicate page content on my website. For example, these pages are being highlighted as duplicated: https://www.ohpopsi.com/photo-wallpaper/made-to-measure/pop-art-graffiti/farm-with-barn-and-animals-wall-mural-3824 and https://www.ohpopsi.com/photo-wallpaper/made-to-measure/animals-wildlife/little-elephants-garden-seamless-pattern-wall-mural-3614. As you can see, both pages are different products, therefore I can't apply a 301 redirect or canonical tag. What do you suggest?
Intermediate & Advanced SEO | | e3creative0 -
Directory with Duplicate content? what to do?
Moz keeps finding loads of pages with duplicate content on my website. The problem is its a directory page to different locations. E.g if we were a clothes shop we would be listing our locations: www.sitename.com/locations/london www.sitename.com/locations/rome www.sitename.com/locations/germany The content on these pages is all the same, except for an embedded google map that shows the location of the place. The problem is that google thinks all these pages are duplicated content. Should i set a canonical link on every single page saying that www.sitename.com/locations/london is the main page? I don't know if i can use canonical links because the page content isn't identical because of the embedded map. Help would be appreciated. Thanks.
Intermediate & Advanced SEO | | nchlondon0 -
Penalized for Duplicate Page Content?
I have some high priority notices regarding duplicate page content on my website www.3000doorhangers.com Most of the pages listed here are on our sample pages: http://www.3000doorhangers.com/home/door-hanger-pricing/door-hanger-design-samples/ On the left side of our page you can go through the different categories. Most of the category pages have similar text. We mainly just changed the industry on each page. Is this something that google would penalize us for? Should I go through all the pages and use completely unique text for each page? Any suggestions would be helpful Thanks! Andrea
Intermediate & Advanced SEO | | JimDirectMailCoach0 -
Avoiding Duplicate Content - Same Product Under Different Categories
Hello, I am taking some past advise and cleaning up my content navigation to only show 7 tabs as opposed to the 14 currently showing at www.enchantingquotes.com. I am creating a "Shop by Room" and "Shop by Category" link, which is what my main competitors do. My concern is the duplicate content that will happen since the same item will appear in both categories. Should I no follow the "Shop by Room" page? I am confused as to when I should use a no follow as opposed to a canonical tag? Thank you so much for any advise!
Intermediate & Advanced SEO | | Lepasti0 -
How do I best handle Duplicate Content on an IIS site using 301 redirects?
The crawl report for a site indicates the existence of both www and non-www content, which I am aware is duplicate. However, only the www pages are indexed**, which is throwing me off. There are not any 'no-index' tags on the non-www pages and nothing in robots.txt and I can't find a sitemap. I believe a 301 redirect from the non-www pages is what is in order. Is this accurate? I believe the site is built using asp.net on IIS as the pages end in .asp. (not very familiar to me) There are multiple versions of the homepage, including 'index.html' and 'default.asp.' Meta refresh tags are being used to point to 'default.asp'. What has been done: 1. I set the preferred domain to 'www' in Google's Webmaster Tools, as most links already point to www. 2. The Wordpress blog which sits in a /blog subdirectory has been set with rel="canonical" to point to the www version. What I have asked the programmer to do: 1. Add 301 redirects from the non-www pages to the www pages. 2. Set all versions of the homepage to redirect to www.site.org using 301 redirects as opposed to meta refresh tags. Have all bases been covered correctly? One more concern: I notice the canonical tags in the source code of the blog use a trailing slash - will this create a problem of inconsistency? (And why is rel="canonical" the standard for Wordpress SEO plugins while 301 redirects are preferred for SEO?) Thanks a million! **To clarify regarding the indexation of non-www pages: A search for 'site:site.org -inurl:www' returns only 7 pages without www which are all blog pages without content (Code 200, not 404 - maybe deleted or moved - which is perhaps another 301 redirect issue).
Intermediate & Advanced SEO | | kimmiedawn0 -
Does duplicate content penalize the whole site or just the pages affected?
I am trying to assess the impact of duplicate content on our e-commerce site and I need to know if the duplicate content is affecting only the pages that contain the dupe content or does it affect the whole site? In Google that is. But of course. Lol
Intermediate & Advanced SEO | | bjs20100 -
Magento Duplicate Content Recovery
Hi, we switched platforms to Magento last year. Since then our SERPS rankings have declined considerably (no sudden drop on any Panda/Penguin date lines). After investigating, it appeared we neglected to No index, follow all our filter pages and our total indexed pages rose sevenfold in a matter of weeks. We have since fixed the no index issue and the pages indexed are now below what we had pre switch to Magento. We've seen some positive results in the last week. Any ideas when/if our rankings will return? Thanks!
Intermediate & Advanced SEO | | Jonnygeeuk0 -
Duplicate content across internation urls
We have a large site with 1,000+ pages of content to launch in the UK. Much of this content is already being used on a .nz url which is going to stay. Do you see this as an issue or do you thin Google will take localised factoring into consideration. We could add a link from the NZ pages to the UK. We cant noindex the pages as this is not an option. Thanks
Intermediate & Advanced SEO | | jazavide0