Getting subdomains unindexed
-
If i turn an application off displaying a 503 error will that get my site unindexed from search engines?
-
Subdomains can be verified as their own site in GWT. Verify the subdomain in GWT, then put a robots.txt on that subdomain excluding the entire subdomain, then request removal in GWT of that entire subdomain. I've had to remove staging and dev sites a couple of times myself.
A couple of things I've found useful in this situation is to make the robots.txt files for both the dev and live sites read only, so you don't accidentally overwrite one with the other when pushing a site live. You can also sign up for a free tool like Pole Position's Code Monitor that will look at the code of a page (including your robots.txt url) once a day and email you if there are any changes so you can fix the file then go hunt down whoever changed the file.
-
GWT was the first placed i checked unfortunately you can only remove directories or pages. I need entire subdomained sites to be removed (in fact they shouldn't of been indexed in the first place).
We use subdomains for our development testing environment when creating client sites and once the site is approved we push it live replacing the old site. Somehow these testing sites are getting indexed and it may pose a threat to duplicate content on different domains. So i am trying to find a solution to get the subdomains (100's of them) unindexed.
I understand a 301 redirect is best but that isn't really applicable since these test sites still need to be reached by clients.
-
With a robots.txt blocking it, you can then go into Google Webmaster Tools and request removal of that particular page or folder from Google's index.
-
No index tag on it works, and putting up a robots.txt that disallows everyone should work as well.
-
Thanks for the quick reply, i will have to try that. Essentially i am trying to get the site un-indexed but i wasn't sure if a 503 would do the trick.
-
Eventually, but that's the code Google recommends to return when your site is having downtime, so I would expect them to be more lenient towards not removing things right away. I wouldn't expect it to be as efficient as returning a 404 or a 410.
The best way to get content de-indexed is to return a page with a meta noindex tag on it, if you're really keen on getting it removed immediately.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How preproduction website is getting indexed in Google.
Hi team, Can anybody please help me to find how my preproduction website and urls are getting indexed in Google.
Technical SEO | | nlogix0 -
Getting mixed signals regarding how Google treats subdomains
All the posts I've read here and elsewhere regarding subdomains come to a similar conclusion, avoid using them because they are treated as a separate site -- and everything that goes along with that. But on my site we have a subdomain on a separate server and it's treated as internal. Also this from Hubspot - "**Use a subdomain of your website like Blog.HubSpot.com. **This is a great idea and this is what we do currently at HubSpot. Many companies have their blog on a subdomain, and it seems to be starting to be somewhat of a standard. The search engines are treating subdomains more and more as just portions of the main website, so the SEO value for your blog is going to add to your main website domain." Any help clarifying this would be greatly appreciated!
Technical SEO | | titleist1 -
Does authority via traffic get passed on to a forwarded domain?
We have purchased a keyword rich domain name as well as a short domain name(the acronym of the company). I would like to only advertise the short version and have a 301 redirect to the longer version. If we get a lot of traffic to the short version is that credit/authority passed on to the domain we are forwarding to?
Technical SEO | | laserclinics0 -
Could getting referral traffic from SEO moz damage your rankings?
Buon Giorno from OS grid reference SE404481 Having just read Googles https://support.google.com/webmasters/bin/answer.py?hl=en&answer=2648487 disavow contnet something troubled me... Is it a possibility you could damage the page rank of a site if you add its url in these posts. Put another way if I added a url pointing to a specific site would googles radar detect the source as SEO and penalise the site in some way? Any insights welcome 🙂
Technical SEO | | Nightwing0 -
How do I get google to index the right pages with the right key word?
Hello I notice that even though I have a site map google is indexing the wrong pages under the wrong key words. As a result its not as relevant and is not ranking properly.
Technical SEO | | ursalesguru0 -
Does hidden text, which appears for an onclick event, get indexed by Google and what SEO impact does this have?
I'm trying to simplify a conversion process with an onclick event to show text rather than having a completely separate page, but wondering if this is going to negatively impact on SEO, especially considering it's hidden text. I've seen a couple of things out there where you could position the text off the screen and the onclick results in it coming on.
Technical SEO | | JuiceBoxOM0 -
I have found this on a site that i have seen many times where can i get one from
Hi i have seen this great map system that i have seen on many sites which i think makes a site look great but i have tried looking for the past few weeks but cannot find where i can get one from. http://www.hypnoslimmer.co.uk/consultant.html does anyone know how these sites do it and where you can get the product from. I use joomla for all my sites Any help would be great
Technical SEO | | ClaireH-1848860 -
Getting rid of low quality
If I wanted to get rid of a batch of low quality pages from the index, Is the best practise to let them 404 and remove them from sitemap files? Thanks
Technical SEO | | PeterM220