Www or no www
-
right now our organization has the a robot text file set to block search engines from indexing the non-www version of our URL - example.com
looks like this
#Dynamically generated robots.txt file for example.com
User-agent: *Disallow: /
Of course, we do allow search engines to crawl the www. version
Is this the best way to do this?
-
Likewise to you Simon, thanks a lot for the advice!
-
Thanks so much Kevin. Much appreciated!
-
I wouldn't recommend setting your robots.txt to disallow indexing of the site without www. The best method is to create a domain wide 301 redirect using .htaccess. You should also set it up in Google Webmaster tools under settings and select with www as your preferred domain. Below is an example of the rewrite in .htaccess.
RewriteEngine on
RewriteCond %{HTTP_HOST} ^mysite.com [NC]
RewriteRule (.*) http://www.mysite.com/$1 [L,R=301]
Hope that helps!
Kevin
-
Hi Kevin
That is a risk, you could end up blocking search engines from indexing your entire website. Usually, both www and non-www point to the same location, so one robots.txt file can apply to both (unless you've configured your site differently).
Check out http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=44231
I'd suggest removing that Disallow command within your robots.txt file and do the following:
Within Google Webmaster Tools, go to 'Site configuration' 'Settings' ~ from there you can set how you'd like your pages to be displayed in Google's search results, either with or without the www. (or can leave Google to decide).
Regards
Simon
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Sites website https://www.opcfitness.com/ title NOT GOOD FOR SEO
We set up a website https://www.opcfitness.com/home on google sites. but google sites page title not good for SEO. How to fix it?
Technical SEO | | ahislop5740 -
From: http://www. to https://
Hi all, I am changing my hosting for legal and SEO reasons from http://www to https:// . Now I hear different stories on the redirects: 1: should i try and change my backlinks? 2: internally all links will be 301 redirected at first. Than I want to (manually) change them. It;s within Wordpress so there should be a plugin for this. Tips? 3: Will it affect my rankings and for what period? What I now know that at first it will drop little but eventually you will rank higher than before. Thanks so much in advance! Tymen
Technical SEO | | Tymen1 -
"non-WWW" vs "WWW" in Google SERPS and Lost Back Link Connection
A Screaming Frog report indicates that Google is indexing a client's site for both: www and non-www URLs. To me this means that Google is seeing both URLs as different even though the page content is identical. The client has not set up a preferred URL in GWMTs. Google says to do a 301 redirect from the non-preferred domain to the preferred version but I believe there is a way to do this in HTTP Access and an easier solution than canonical.
Technical SEO | | RosemaryB
https://support.google.com/webmasters/answer/44231?hl=en GWMTs also shows that over the past few months this client has lost more than half of their backlinks. (But there are no penalties and the client swears they haven't done anything to be blacklisted in this regard. I'm curious as to whether Google figured out that the entire site was in their index under both "www" and "non-www" and therefore discounted half of the links. Has anyone seen evidence of Google discounting links (both external and internal) due to duplicate content? Thanks for your feedback. Rosemary0 -
Domain Structure - without www.
I'm working on a new project and we would prefer to not use the www. - for name/branding reasons. Are there any SEO ramifications from setting the domain without the www and using 301 redirects for all home page extensions to forward to -> domain.com(without the www)? Furthermore, we will be hosting many profiles on this site and would like to structure them for optimal SEO. Would there be an issue with using sub domains - user.domain.com, or would sub directories be more optimal? Thank you in advance!
Technical SEO | | NickMacario0 -
WWW wordpress multisite network for Main
I have a year old site with not many links, and my original intent was non www so http://rootdomain.com However, since its a subdomain multisite, I have received notice from google webmaster tools because of poor quality on one of my subsites.maindomain.com I do have the google account setup to be "root domain only" - however, i am still being penalized it seems. So should I convert my wordpress main site to use WWW. and modify my google webmaster/analytics profile, update my site links, etc. Or what do you suggest?
Technical SEO | | inllc0 -
To 301 redirect or not to 301 redirect? duplicate content problem www.domain.com and www.domain.com/en/
Hello, If your website is getting flagged for duplicate content from your main domain www.domain.com and your multilingual english domain www.domain.com/en/ is it wise to 301 redirect the english multilingual website to the main site? Please advise. We've recently installed the joomish component to one of our joomla websites in an effort to streamline a spanish translation of the website. The translation was a success and the new spanish webpages were indexed but unfortunately one of the web developers enabled the english part of the component and some english webpages were also indexed under the multilingual english domain www.domain.com/en/ and that flagged us for duplicate content. I added a 301 redirect to redirect all visitors from the www.domain/en/ webpages to the main www.domain.com/ webpages. But is that the proper way of handling this problem? Please advise.
Technical SEO | | Chris-CA0 -
Site:www.tld.com rank is it a measure of googles per page importance?
Hello, does the order of pages in a site:www.tld.com search show how important each page is to google? what if the homepage is not the first result?
Technical SEO | | adamzski0 -
Domain with or without "www"
Does it influence the search engine result if we have our domain name without the "www." ?
Technical SEO | | netbuilder0