Homepage duplicated content - 301 redirection - "Rel="canonical" - Headache
-
Hello Moz community,
I already have 301 redirection in my .htaccess , just like this:
RewriteEngine On
RewriteCond %{HTTP_HOST} ^mysite.com$
RewriteRule ^(.*)$ "http://www.mysite.com/$1" [R=301,L]Mozpro shows that I have duplicated content for www.mysite.com & www.mysite.com\index.html
I am a little lost, there is " 301 Redirect" and "Rel="canonical", moz replied me back that I need to use "Rel="canonical", but can one use both together?
i also read on forums, if I 301 redirect index to /, there will be infinite loop and such.
Another story, I am also getting IP Canonicalization error when I checked my SEO performance:
Your site's IP ...* does not redirect to your site's domain name. This could cause duplicate content problems if a search engine indexes your site under both its IP and domain name.**
Which adds to my headache. What is this one all about?
There is no single thread or page here, to clear all this confusion and give a straight solution, but rather each page or thread says do a different thing.
I am lost, so somebody help!
-
Hi
here is how
http://www.webconfs.com/how-to-redirect-a-webpage.php
http://moz.com/learn/seo/redirection
To determine rather not you should redirect to www. or no-www
use http://moz.com/researchtools/ose
to figure out which has more back links pointing to it then 301 redirect to that.
In terms of if one is better than the other there is really not a difference however if they are both equal meeting having no back links or identical.
You might want to go with www. simply because having the subdomain allows you to do some more things like add Google page speed other miscellaneous things in the future.
But really go with the one with the most links if there are no links then choose the one you want going to Google webmaster tools to make sure you add both
to Google webmaster tools than pick the same URL you chose to redirect to as your main URL Google will then only send Google bot out looking for that URL
Here
https://support.google.com/webmasters/answer/44231?hl=en
Hope this helps,
Tom
-
I was just about to ask the same question and about to do a 301 redirect myself. Mine question is should I use www. or just http://site.com? I know its not good to have both as it could be cited as duplicate content, but what do you all think? My site is ***techhelp.fake (not really .fake and its not a TLD or whatever.
-
It could be cache
Try
http://codeblow.com/questions/getting-rid-of-index-html-from-url-and-adding-www-with-a/
try RewriteEngine On
RewriteCond %{HTTP_HOST} ^mydomain.com$ [NC]
RewriteRule ^(.*)$ http://www.mydomain.com/$1 [R=301,L]Redirect 301 /index.htm http://www.mydomain.com
-
If you are still having problems could you send me your domain either in the form here or privately?
I am psyched that the redirect worked that is really good.
These other issues I can troubleshoot for you very quickly using some tools I have in addition to the excellent tools Moz has.
Download a free copy of screaming frog just a have it is honestly worth it as it is free and awesome.
Can I be of more assistance,
Tom
-
Nice!
Yes this will check it 2 times a day if needed
-
The
duplicated warning for www.mysite.com and www.mysite.com/index.html
will stay for a week
Or unfortunately you would have to delete your campaign or start a new campaign that would let you because moz indexes and checks for those errors once week it would need a new campaign but I would not worry about the errors that showing you now that you have corrected this
Awesome excited about redirect work. I did not know what to tell you if that did not. Work
Okay you can crawl your domain and get very similar results almost the same thing. Using this
https://moz.com/researchtools/crawl-test
If your site is under 500 pages and you want quick confirmation in addition you can use this tool for free up to 500 pages
http://www.screamingfrog.co.uk/seo-spider/
So use this tool
If you want I can verify it as well for you
https://moz.com/researchtools/crawl-test
If it is your first time using the Moz campaign tool and you have no history to worry about you can simply start a new campaign. In addition to these tests
I hope this helps,
Tom
-
Just checked my site at http://www.siteliner.com/
Still see duplicated warning for www.mysite.com and www.mysite.com/index.html
Headache :d
-
removed and added RewriteEngine On RewriteRule ^index.html$ / [R=301,L]
now index.html is redirected to / successfully.
How can I ask Mozpro to check again? I can still see duplicate content warning in my Mozpro account.
Should I go for Crawl test menu?
-
Ok are you on a CMS what type of code?
removing the code and placing this in RewriteEngine On RewriteRule ^index.html$ / [R=301,L]
http://www.webconfs.com/how-to-redirect-a-webpage.php
http://moz.com/blog/apache-redirect-an-index-file-to-your-domain-without-looping
http://moz.com/learn/seo/redirection
We we have to add the removed the /
See this as well
http://dense13.com/blog/2012/10/29/removing-index-html-with-mod_rewrite-in-htaccess/
RewriteEngine On
RewriteCond %{HTTP_HOST} ^www.mydomain.com$ [NC]
RewriteRule ^(.*)$ http://mydomain.com/$1 [R=301,L]Redirect 301 /index.htm http://mydomain.com
-
Thank you Thomas for your time and answers.
I added the following code to my htaccess file, but then could not access my site:
Options +FollowSymLinks RewriteEngine On RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.*index\.html?\ HTTP/ RewriteRule ^(.*)index\.html?$ http://www.mydomain.com/$1 [R=301,L]
This is the error when I checked my site:
Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the server administrator, webmaster@********** and inform them of the time the error occurred, and anything you might have done that may have caused the error.
More information about this error may be available in the server error log.
Additionally, a 500 Internal Server Error error was encountered while trying to use an ErrorDocument to handle the request.
checking with my hosting company about the IP error in the meantime.
-
Hi,
I am going to try to make this as easy as possible before look by pasting your questions in italics and answering and non-italics.
I believe that you are new server setup if that is the case is in need of some basic setup
in addition to correctly using the 301 redirect as well as the canonical tag to fix your issues.
I am a little lost, there is " 301 Redirect" and "Rel="canonical", moz replied me back that I need to use "Rel="canonical", but can one use both together?
i also read on forums, if I 301 redirect index to /, there will be infinite loop and such.
this is very important for you to read
http://moz.com/blog/apache-redirect-an-index-file-to-your-domain-without-looping
This is also an excellent reference
http://moz.com/learn/seo/canonicalization
** I cited this to save you time from the page below **
https://support.google.com/webmasters/answer/139066?hl=en
"Use 301 redirects for URLs that are not canonical
Suppose your page can be reached in multiple ways:
http://example.com/home
http://home.example.com
http://www.example.com
It's a good idea to pick one of those URLs as your preferred (canonical) destination, and use 301 redirects to send traffic from the other URLs to your preferred URL. A server-side 301 redirect is the best way to ensure that users and search engines are directed to the correct page. The 301 status code means that a page has permanently moved to a new location."
http://moz.com/blog/canonical-url-tag-the-most-important-advancement-in-seo-practices-since-sitemaps
http://googlewebmastercentral.blogspot.com/2009/02/specify-your-canonical.html
You want to place the canonical tag on the URL you wish to have search engines recognize as your home URL
Yes they can be used together if they are both pointing to the same place it is not that they have an issue.
The canonical tag tells Google this is the page I want whichever the canonical tag is pointing to.
Mozpro shows that I have duplicated content for www.mysite.com & www.mysite.com\index.htm
The correct Regex to accomplish that just removed removed the index HTML file on your homepage
<code class="htaccess" title="in your .htaccess file">RewriteEngine On RewriteRule ^index\.html$ / [R=301,L]</code>
this would remove it from your entire site
<code class="htaccess" title="in your .htaccess file">RewriteEngine On RewriteRule ^index\.html$ / [R=301,L] RewriteRule ^(.*)/index\.html$ /$1/ [R=301,L]</code>
This is exactly what I think you need to set it up correctly
In an existing .htaccess file or a new .htaccess file add the following code:
http://www.smartlabsoftware.com/howto/redirect-index-page.htm
Options +FollowSymLinks
RewriteEngine On
redirect index.htm and index.html to / (do this before non-www to www)
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.index.html?\ HTTP/
RewriteRule ^(.)index.html?$ http://www.mydomain.com/$1 [R=301,L]redirect mydomain.com -> www.mydomain.com
RewriteCond %{HTTP_HOST} ^mydomain.com$ [NC]
RewriteRule ^(.*)$ http://www.mydomain.com/$1 [R=301,L]Depending on entirely with the server you are currently running this could be of use as well.
http://css-tricks.com/how-to-redirect-indexhtml-to-indexphp/
.htaccess ReWrite
**`RewriteEngine On RewriteBase / RewriteRule ^index\.html?$ / [NC,R,L]`**
Your site's IP ..*. does not redirect to your site's domain name. This could cause duplicate content problems if a search engine indexes your site under both its IP and domain name.
Which adds to my headache. What is this one all about?
you need to set your host file and server IP catch all it is illustrated beautifully in this
here is the short version
when you are setting up your server rather at your hosting company or yourself you need to set what is called a "catch all" this prevents the IP address acting like a domain.
As well as preventing people from just using their DNS to point their domain to your site and have it resolved. It is an extremely fundamental and important part of setting up a server correctly. You want to be certain that your catch all
host name being set up in a local database this Should be done by your hosting company if there managing the server however if it is a non managed server I recommend you follow the steps below.
# Host Database # # localhost is used to configure the loopback interface # when the system is booting. Do not change this entry. ## 127.0.0.1 localhost #Virtual Hosts 12.34.56.789 example.com
this means your IP address corresponds with your domain and only your domain.
I hope I have answered your questions please let me know if I can be of assistance with any more.
this is a great tutorial on setting this up correctly if you are using a VPS that is non managed again call your hosting company if it is a managed server.
Sincerely,
Thomas
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Duplicate Content with TranslatePress Plugin
Hello, I am using a plugin named translatepress, however whenever I activate it the moz crawl gives my site as having duplicate content on it. How do I avoid this? I would like to have a website version in English and Portuguese? Thanks
Getting Started | | SilvinoFernandes0 -
How to show the KD in "All Keyword Suggestions"
Hi Moz team, I wanna show KD as a column on the "All Keyword Suggestions" page. So I do not need to research every keyword that I want to focus on. Is there any way do this? Thanks, Bruce
Getting Started | | Bruce.L0 -
Problem with duplicate titles
Hi there, after a site crawl Moz has reported that I have duplicated titles, however I don't understand why or how to fix it. Basically the urls are for the same collection or product and are identical except one has a -1 on the end of it or sometimes a %20. I cannot delete the url as they don't exist in my shopify store so cannot find them. The relevant page has the correct url when I check in shopify. Anyone have any ideas?
Getting Started | | cookie64980 -
Can't track my site, keep getting "Ooops. Our crawlers are unable to access that URL"
Hello, So i keep getting this message and I went to hurl.it and I get 200 response. But it appears its not my actual homepage bc it says the body is empty and in the title it says "COMING SOON" which is not what my actual homepage says. Does anyone know what this means?? Thank you in advance! Rena
Getting Started | | Palila-Studio0 -
I'm setting up a new campaign and getting the error "This does not appear to be a valid URL" What's wrong?
I've tried multiple times (over 2 days) with every variation of the URL with no luck. Any ideas for why the URL does not seem to be working?
Getting Started | | jgrammer0 -
How to make site pages appear higher than homepage
Hi, our site sells a mixture of clothes, for example jackets, hats, scarfs and gloves. when somebody searches for 'hats in Chicago' our main website would appear. How can we make it so that our webpage with our hats appears? Thanks
Getting Started | | danieldunn100 -
What is the "other" group in traffic to your site on the moz dashboard ?
What is the "other" group in traffic to your site on the moz analytics dashboard ? "Not provided" is in search - organic search history so it's not that.
Getting Started | | Crocodesign0