Will password protecting my test sub-domain help keep the SEs from indexing it?
-
Hi, all. I'm working in an unfamiliar area here, so I hope someone can tell me if I'm out in left field.
I am building a sub-domain called http://test.mysite.com, so that I can upload a client's still-under-construction site while working on it. When completed, it'll go up on his server, replacing his old site. Obviously, I want to ensure that it doesn't get indexed while it's on my test platform.
A friend suggested that I password it with htaccess and htpasswd, since we can never be certain the SEs will obey site directives.
My question is, what do you think would be the best (and hopefully, simplest) way to accomplish this?
I'm no code-monkey, so "simple" is a big plus!
Doc
By the way, the platform will be Wordpress CMS.
-
A different Matt but I have to/still agree that you need to password protect the site. This isn't just for a protection against crawlers but also anyone else who might be snooping around. Unless your client is okay with their work being released early into the wild you should password protect it.
The good news is that many hosting companies have tools that will automagically generate the .htaccess files for your.
-
Thanks, Darryl-
Passwording the site seemed like a good option, although I wasn't aware that Matt had ever stated that. That being the case, it would certainly seem like the way to go. Thanks for the input!
-
Also, a good way to go is the following:
- tell search engines to go away in robots.txt
- to insert a meta noindex tag
- block in .htaccess as well
Matt Cutts stated that the only 100% sure way is to password protect the folder
-
Thanks for the response, Matt. So you feel like that's a sure way? There seems to be some different opinions on whether or not all the SEs will respect that. I had always thought it was a solid way to do it,too. But some of the arguments I'm hearing have me in doubt, now.
-
htaccess is a very simple way to protect the site from crawlers. If they can't access the pages they certainly can't index them.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why is Google not indexing my site?
I'm a bit confused as to why my site just isn't indexing on Google. Even if I type in my brand name, my social channels rank and there's no evidence of my website. I've followed all of the advice I've read and gone into webmaster tools and got the Wordpress yoast plug-in but nothing seems to be making a difference!One thing I've noticed, in Google Webmaster Tools it says "Couldn’t communicate with the DNS server." in site errors. I've called GoDaddy and they said that everything is fine. A bit frustrating. Trying to work out what my next steps should be but feeling a bit lost to be honest! Any help GREATLY appreciated!
Technical SEO | | j1066s0 -
Pages Indexed Not Changing
I have several sites that I do SEO for that are having a common problem. I have submitted xml sitemaps to Google for each site, and as new pages are added to the site, they are added to the xml sitemap. To make sure new pages are being indexed, I check the number of pages that have been indexed vs. the number of pages submitted by the xml sitemap every week. For weeks now, the number of pages submitted has increased, but the number of pages actually indexed has not changed. I have done searches on Google for the new pages and they are always added to the index, but the number of indexed pages is still not changing. My initial thought was as new pages are added to the index, old ones are being dropped. But I can't find evidence of that, or understand why that would be the case. Any ideas on why this is happening? Or am I worrying about something that I shouldn't even be concerned with since new pages are being indexed?
Technical SEO | | ang1 -
Indexing Problem
My URL is: www.memovalley.comWe have submitted our sitemap last month and we are having issues seeing our URLs listed in the search results. Even though our sitemaps contain over 200 URLs, we only currently only have 7 listed (excluding blog.memovalley.com).Can someone help us with this? | |
Technical SEO | | Memovalley
| | | | It looks like Googlebot has timed out, at least once, for one of our URLs. Why is Googlebot timing out? My server is located at Amazon WS, in North Carolina and it is a small instance. Could Google be querying multiple URLs at the same time and jamming my servers? Could it be becauseThanks for your help!0 -
Domain recommendations for non indexed sites
For complicated internal reasons (which I swear have validity) we are starting websites that will then disallow from Google for ~1 year. Is there any value in purchasing "premium" domains with SEO juice or will we lose the value in the time we are hiding the website from Google?
Technical SEO | | theLotter0 -
Transfer a Main Domain to a Sub-Domain
My IT department tells me they want to transfer my main site domain, which has been in existence since 1999 as an e-commerce site (maindomain.com) to a sub-domain (www2.maindomain.com) or a completely new domain (newdomain.net). This is because we are launching a new website and B2C e-commerce engine, but we still have to maintain the legacy B2B e-commerce engine which contains hard-coded URLs, and both systems can't use the same domain. I've been researching the issue across SEOmoz, but I haven't come across this exact type of scenario (mostly I've seen a sub-domain to new domain). I see major problems with their proposal, including negative SEO impact, loss of domain authority/ranking and issues with branding. Does anyone know the exact type of impact I can expect to see in this scenario and specific steps I should go about to minimize the impact? Btw, I will be using Danny Dover's guide on properly moving domains where appropriate. Thanks!
Technical SEO | | AscendLearning0 -
Old Domain - What to do?
A client recently bought an older domain that is keyword-rich to an aspect of his company. The main website has both e-commerce and call-to-action elements. Our team is split on whether or not to create a micro-site on that domain focused on that aspect of the work that he does or to simply redirect the old domain to his main website. I have not had the opportunity to look at the link profile of the recently acquired domain nor do I have any idea of how many times it's changed hands (which would seem to now be a possible indicator of doorway pages). If any clarification would help, please let me know and I'll do my best to answer.
Technical SEO | | MountainMedia0 -
Subfolder to Root Domain, Should i or not?
Dear All, I am kinda of stuck and need your helps, I have major client's site which is located in subdirectory. Instance: http://www.majorclient.com.cn/cn/ and as you can see domain name is already localized and they are not planning to make any english version of the site, So there will be no /eng/ folder but clients whole website is /cn/ subdirectory which is totally unnecessary and i would like to 301 whole site to **http://www.majorclient.com.cn ** But the problem is that this site has been /cn/ folder forever since the beginning and lot of trusted links are pointing to www.majorclient.com.cn/cn/ So should i move it or just configure 301 www.majorclient.com.cn to www.majorclient.com.cn/cn/ leave it there and don't bother. Help?
Technical SEO | | DigitalJungle0 -
Sub-Domain Choice Dilemma
We have successfully rolled out 5 sub-domains using very industry specific KWs as the sub, e.g. familylawyers. We're rolling our an employment focused sub.and ideally would use employmentlawyers.XXX.com. However I'm tempted to use a long established (5+ years) sub-domain with a topic related KW that now hosts a non-active blog with PR3 - employment-law.XXXX.com. It has 49 indexed pages, some with PR2. So there's potential for getting a kickstart on traffic and trust with some redirects. Should I go for instant gratification or build for the long haul with the slightly more beneficial URL KWing? I should add that this sub-domain will have thousands of pages that are geo and sub-category focused - a typical URL would be. sub-domain.XXX.com/TX/Austin/wrongful-termination-lawyers.html THX for any opinions.
Technical SEO | | legalseo0