Rel canonical and duplicate subdomains
-
Hi,
I'm working with a site that has multiple sub domains of entirely duplicate content. So, the production level site that visitors see is (for made-up illustrative example):
Then, there are sub domains which are used by different developers to work on their own changes to the production site, before those changes are pushed to production:
Google ends up indexing these duplicate sub domains, which is of course not good.
If we add a canonical tag to the head section of the production page (and therefor all of the duplicate sub domains) will that cause some kind of problem... having a canonical tag on a page pointing to itself? Is it okay to have a canonical tag on a page pointing to that same page?
To complete the example...
In this example, where our production page is 123abc456.edu, our canonical tag on all pages (this page and therefor the duplicate subdomains) would be:
Is that going to be okay and fix this without causing some new problem of a canonical tag pointing to the page it's on?
Thanks!
-
Hi Bob,
That excellent question I'll have to look in to and confirm. More later. Thanks!
-
Is the subdomain data stored on the server as directories?
So for example, is the Moe.123abc456.edu data stored in a folder like 123abc456.edu/Moe
If so, you can simply have one robots.txt on your root domain, blocking those directories
Disallow: /Moe/
-
Well, Bob, it looks like you're right! I guess it will for sure see all the pages in
as the ones to remove and not
Also, how does that robots text not get pushed to production as the developer working on that branch completes his work and pushes it to production.
I must confess, it still feels a little like bomb disposal.
-
This should be exactly what you need: http://support.google.com/webmasters/bin/answer.py?hl=en&answer=1663427
-
Hi Bob,
Thanks for the suggestion/question. I'm thinking about that, but wouldn't putting some robots do not crawl text on pages already indexed be a little like closing the barn door after the horses left? Do you think it would un-index the already crawled sub-domain? Thanks!
-
Assuming that you do not need the development environments indexed in Google, why not simply block all crawlers on those subdomains?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Hundreds of Subdomains under a powerful domain
Hello, I own a domain FeedsPortal - com As you can see from the link profile, it has some fantastic referring domains and links. Because of this, it has a DA of 86, with a good CF/TF. The problem is that nearly all of these powerful links are for the sub domains under the main domain. For example, it has a link on this page on MSN... https://www.msn.com/en-us/news/offbeat/finland-home-of-the-dollar103000-speeding-ticket/ar-AA9GA9i?ocid=ansAtlantic11 On this MSN article, it has a link to http://feedproxy.google.com/~r/TheAtlantic/~3/PK4bw0Tkkps/story01.htm (a link under Google.com), which then forwards to my domain under a sub domain http://theatlantic.feedsportal.com/c/34375/f/..... I have many hundreds of sub domains like this. I have a feeling redirecting all non-existent sub domains to the homepage would be a bad idea for SEO. Does anyone else see of a way to do this without harming my SEO? I suppose the only way to do it properly would be to write articles about each subdomain. For example, http://theatlantic.feedsportal.com, write an article about The Atlantic, then forward all traffic meant for theatlantic.feedsportal.com to feedsportal.com/10-reasons-why-the-atlantic-is-great/ Does anyone else have an idea of how to at least get a list of the non-existant sub domains that have links so I can maybe create articles for each sub domain? Or is there a simpler way to do this. Thanks!
Intermediate & Advanced SEO | | thinkingdif0 -
Competition site with Duplicate Name
I have a client who is in real estate and one of his competition used the same Name and Domain and added the word Toronto in front. The competition is putting similar content on the website. The competition also hijacked the Google listing since they both work in the same building. Is this going to affect the ranking our website? What negative effect will it put on our website?
Intermediate & Advanced SEO | | KaranX0 -
Duplicate content issue
Hello! We have a lot of duplicate content issues on our website. Most of the pages with these issues are dictionary pages (about 1200 of them). They're not exactly duplicate, but they contain a different word with a translation, picture and audio pronunciation (example http://anglu24.lt/zodynas/a-suitcase-lagaminas). What's the better way of solving this? We probably shouldn't disallow dictionary pages in robots.txt, right? Thanks!
Intermediate & Advanced SEO | | jpuzakov0 -
Duplicate page wordpress title
Hi Moz Fan, I have a few question that confuse by now, According to my website used wordpress for create a blog.
Intermediate & Advanced SEO | | ASKHANUMANTHAILAND
but now i have got duplicate page title from wordpress blog and very weird because these duplicate
page title look like these term Lamborghini ASK Hanuman Club < title that was duplicate but they duplicate with the page like this
articles/13-วิธีง่ายๆ-เป็น-คนรวย-ได้/lamborghini-2/ , /articles/13-วิธีง่ายๆ-เป็น-คนรวย-ได้/lamborghini/ I don't know why Worpress create a new page that only have 1 pics, So I think that might be the effect
from Wordpress PHP Code Or wrong setting. Can anyone suggestion what should we do for these issues ?
because i have many duplicate page title from this case.1 -
Rel="canonical" and rel="alternate" both necessary?
We are fighting some duplicate content issues across multiple domains. We have a few magento stores that have different country codes. For example: domain.com and domain.ca, domain.com is the "main" domain. We have set up different rel="alternative codes like: The question is, do we need to add custom rel="canonical" tags to domain.ca that points to domain.com? For example for domain.ca/product.html to point to: Also how far does rel="canonical" follow? For example if we have:
Intermediate & Advanced SEO | | AlliedComputer
domain.ca/sub/product.html canonical to domain.com/sub/product.html
then,
domain.com/sub/product.html canonical to domain.com/product.html0 -
What is the proper syntax for rel="canonical" ??
I believe the proper syntax is like this [taken from the SEOMoz homepage]: However, one of the sites I am working on has all of their canonical tags set up like this: I should clarify, not all of their canonicals are identical to this one, they simply use this naming convention, which appears to be relative URLs instead of absolute. Doesn't the entire URL need to be in the tag? If that is correct, can you also provide me with an explanation that I can give to management please? They hate it when I say "Because I said so!" LOL
Intermediate & Advanced SEO | | danatanseo0 -
Canonical URLs and Sitemaps
We are using canonical link tags for product pages in a scenario where the URLs on the site contain category names, and the canonical URL points to a URL which does not contain the category names. So, the product page on the site is like www.example.com/clothes/skirts/skater-skirt-12345, and also like www.example.com/sale/clearance/skater-skirt-12345 in another category. And on both of these pages, the canonical link tag references a 3rd URL like www.example.com/skater-skirt-12345. This 3rd URL, used in the canonical link tag is a valid page, and displays the same content as the other two versions, but there are no actual links to this generic version anywhere on the site (nor external). Questions: 1. Does the generic URL referenced in the canonical link also need to be included as on-page links somewhere in the crawled navigation of the site, or is it okay to be just a valid URL not linked anywhere except for the canonical tags? 2. In our sitemap, is it okay to reference the non-canonical URLs, or does the sitemap have to reference only the canonical URL? In our case, the sitemap points to yet a 3rd variation of the URL, like www.example.com/product.jsp?productID=12345. This page retrieves the same content as the others, and includes a canonical link tag back to www.example.com/skater-skirt-12345. Is this a valid approach, or should we revise the sitemap to point to either the category-specific links or the canonical links?
Intermediate & Advanced SEO | | 379seo0 -
Robots.txt disallow subdomain
Hi all, I have a development subdomain, which gets copied to the live domain. Because I don't want this dev domain to get crawled, I'd like to implement a robots.txt for this domain only. The problem is that I don't want this robots.txt to disallow the live domain. Is there a way to create a robots.txt for this development subdomain only? Thanks in advance!
Intermediate & Advanced SEO | | Partouter0