Www or no www
-
right now our organization has the a robot text file set to block search engines from indexing the non-www version of our URL - example.com
looks like this
#Dynamically generated robots.txt file for example.com
User-agent: *Disallow: /
Of course, we do allow search engines to crawl the www. version
Is this the best way to do this?
-
Likewise to you Simon, thanks a lot for the advice!
-
Thanks so much Kevin. Much appreciated!
-
I wouldn't recommend setting your robots.txt to disallow indexing of the site without www. The best method is to create a domain wide 301 redirect using .htaccess. You should also set it up in Google Webmaster tools under settings and select with www as your preferred domain. Below is an example of the rewrite in .htaccess.
RewriteEngine on
RewriteCond %{HTTP_HOST} ^mysite.com [NC]
RewriteRule (.*) http://www.mysite.com/$1 [L,R=301]
Hope that helps!
Kevin
-
Hi Kevin
That is a risk, you could end up blocking search engines from indexing your entire website. Usually, both www and non-www point to the same location, so one robots.txt file can apply to both (unless you've configured your site differently).
Check out http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=44231
I'd suggest removing that Disallow command within your robots.txt file and do the following:
Within Google Webmaster Tools, go to 'Site configuration' 'Settings' ~ from there you can set how you'd like your pages to be displayed in Google's search results, either with or without the www. (or can leave Google to decide).
Regards
Simon
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Redirect our www.website.com to website. Com/target-keyword
Dear moz community I have been analyzing the websites that rank in top 20 for our target keywords. All of the top 20 sites except us have their websites re directing to websites.com/target keyword. This is due to probably because they have multiple city's and one of the target keywords term is cityname + word. My question is and idea - make a 301 to our www.website.com to /city-keyword and start linking to that page with new links. Would that bring any benefit? Seems that it's a very strong ranking signal. Any threats that I must take into account? We currently rank as #9
Technical SEO | | advertisingcloud0 -
I have made my new website live. But while checking in Google it is not showing in search result ( site: www.oomfr.com ). Can anybody please advice.
Hi Team, I have made my new website live. But while checking in Google it is not showing in search result ( site: www.oomfr.com ). Can anybody please advice.
Technical SEO | | nlogix0 -
Need Help With WWW vs. Non-WWW Duplicate Pages
A friend I'm working with at RedChairMarket.com is having duplicate page issues. Among them, both www and non-www URLs are being generated automatically by his software framework, ASP.net mvc 3. How should we go about finding and tackling these duplicates? Thanks!
Technical SEO | | BrittanyHighland0 -
Searching in Google using the Site:www.example.com specification - is it in an order?
Hi Gurus, Just a quick searching question. If you do a Google search using the site: specification eg. site:www.example.com Is the list returned by Google in an order of something similar to 'Page Authority' or some other order eg. page first seen date etc. Because you are looking at your single site, is Google listing your pages back to you in it's perceived order of current 'popularity'? Thanks, Brad
Technical SEO | | BM70 -
301 Re Direct Question for www
Can smeone check this code to make sure it is right so thaqt my site uses www always. RewriteCond %{HTTP_HOST} ^exercisebiology.com [NC] RewriteRule ^(.*)$ http://www.exercisebiology.com/$1 [R=301,L] I had the hostgators customer service personnel perform this. But I cannot get it to redirect. But he says it works.
Technical SEO | | anoopbal0 -
Different levels of PA without "www"?
Hello Guy´s! The last days I have been surprised to see that the levels of PA, mR, and mT vary when it is or not the "www" in the URL: 1. http://patagonline.com/viajes-argentina 2. http://www.patagonline.com/viajes-argentina Both URL's correspond to the keyword "viajes Argentina" our incoming links... In this case, it´s convenient to do a 301redirect from URL 1 to 2? Thanks a lot for your help!!
Technical SEO | | maty0 -
For SERPS which is better www or non www
Does it matter if i use to www or no www for SERPS on google? Obviously the redirection would be set up for either in the htaccess file.
Technical SEO | | apwade0 -
Duplicate Homepage: www.mysite.com/ and www.mysite.com/default.aspx
Hi, I have a question regarding our client's site, http://www.outsolve-hr.com/ on ASP.net. Google has indexed both www.outsolve-hr.com/ and www.outsolve-hr.com/default.aspx creating a duplicate content issue. We have added
Technical SEO | | flarson
to the default.aspx page. Now, because www.outsolve-hr.com/ and www.outsolve-hr.com/default.aspx are the same page on the actual backend the code is on the http://www.outsolve-hr.com/ when I view the code from the page loaded in a brower. Is this a problem? Will Google penalize the site for having the rel=canonical on the actual homepage...the canonical url. We cannot do a 301 redirect from www.outsolve-hr.com/default.aspx to www.outsolve-hr.com/ because this causes an infinite loop because on the backend they are the same page. So my question is two-fold: Will Google penalize the site for having the rel=canonical on the actual homepage...the canonical url. Is the rel="canonical" the best solution to fix the duplicate homepage issue on ASP. And lastly, if Google has not indexed duplicate pages, such as https://www.outsolve-hr.com/DEFAULT.aspx, is it a problem that they exist? Thanks in advance for your knowledge and assistance. Amy0