Hi,

I have several subdomains that present no real SEO value, but are being indexed. They don't earn any backlinks either. What's the best way of cleaning them up?

I was thinking the following:

1. Verify them all in Webmaster Tools.

2. Remove all URLs from the index via the Removal Tool in WMT

3. Add site-wide no-index, follow directive.

Also, to remove the URLs in WMT, you usually have to block the URLs via /robots.txt.

If I'd like to keep Google crawling through the subdomains and remove their URLs, is there a way to do so?

Hi,

We had a coding update and a side-effect of that was that our directive was emptied, in other words it now reads as:

on all of the site.

I've since noticed that Google's cache date on all of the pages - at least, the ones I tested - have a Cached date of no later than 17 December '12 - that's the Monday after the directive was removed on mass.

So, A, does anyone have solid evidence of an empty directive causing problems? Past experience, Matt Cutts, Fishkin quote, etc.

And then B - It seems fairly well correlated but, does my entire site's homogenous Cached date point to this tag removal? Or is it fairly normal to have a particular cache date across a large site (we're a large ecommerce site).

Our site: http://www.zando.co.za/

I'm having the directive reinstated as soon as Dev permitting.

And then, for extra credit, is there a way with Google's API, or perhaps some other tool, to run an arbitrary list and retrieve Cached dates? I'd want to do this for diagnosis purposes and preferably in a way that OK with Google. I'd avoid CURLing for the cached URL and scraping out that dates with BASH, or any such kind of thing.

Cheers,