I figured .htaccess would be the best route. Thank you for researching and confirming. I appreciate it.

Hi Tim,

Assigning a noindex tag to these files will not block them, only prevent them from showing in SERPs. This is the intended goal and the reason I deleted my robots.txt file which prevented crawling.

Unless I'm severely misreading the links provided, which I've read before, it seems Google is stating that they read, render, and sometimes index .CSS and .JS files. Here's an article written a week after the second article you posted.

The aforementioned WordPress plugin and theme files hosted on my server are indeed showing up in Google SERPs.

I do not want to prevent Googlebot from reaching these files as they're needed for optimal site performance, but I do want them to be no-indexed. Thus, I don't want robots.txt to prevent crawling, only indexing.

Let me know if I'm misunderstanding.

Hi,

Google has indexed a few .CSS and .JS files that belong to our WordPress plugins and themes. I had them blocked via robots, but realized this doesn't prevent indexation (and can likely hurt us since Google wants to access these files).

I've since removed the robots instructions, submitted a removal request via Search Console, but want to make sure they don't come back.

Is there a way to put a noindex tag within .CSS and .JS files? Or should I do something with .htaccess instead?

Yup. I just saw the article. Looks like they actually used my screenshot on it from last year. Glad to see they were punished. I hate when giant sites get a free pass on stuff like this.

Sorry, the link is definitely follow and as you noted, an obvious exact match anchor. They're a huge site and are obviously aware of what they're doing. The link itself goes to the person's profile page and not a generic category page which is still pretty risky.

Hi,

We have a business registered on Thumbtack so we receive their newsletters. I'm aware that review sites offering a "badge" or verification logo which links back to your profile is nothing new. But the email I received from Thumbtack is a fairly blatant attempt to game Google for popular keywords.

I was just curious on your thoughts about this. I believe it was Overstock who did something like this and got slapped by Google pretty hard for a while. Could Thumbtack be heading down the same path?

Image: http://i.imgur.com/FWPnmEP.jpg

Right, but having lots of 404's that are still indexed probably isn't good for your site in general. If you wanted them de-indexed, 301'ing them to a new folder and filing a single removal request for that entire directory would probably work.

Thanks for the help. I've heard from a few people that they will recrawl these pages again even if nothing is linking to them. That's reassuring. Thanks all.

Interesting. Any reason why you haven't simply filed a removal request? I feel if there's too many to manually do, you could 301 them to a specific directory and then manually remove that directory all at once?

Hi Martijn,

Thanks for the response. I must apologize as I left out an important detail. While are pages are "No results" and basically useless to the user, they're not actually 404'd pages. They're live, valid pages that basically offer nothing.

As I stated earlier, 404'ing them would be ideal for us if we could be sure Google would recrawl them. I am hesitant due to uncertainty of Googlebot re-crawling unlinked internal links. Our deeper pages like these have not been updated/recrawled yet, so I'm a bit unsure as to how likely they will.

I guess I should just go ahead and 404 all of them now and see what happens, since it can't hurt. Just curious about Googlebot in general since it always helps to know more!

We accidentally introduced Google to our incomplete site. The end result: thousands of pages indexed which return nothing but a "Sorry, no results" page. I know there are many ways to go about this, but the sheer number of pages makes it frustrating.

Ideally, in the interim, I'd love to 404 the offending pages and allow Google to recrawl them, realize they're dead, and begin removing them from the index. Unfortunately, we've removed the initial internal links that lead to this premature indexation from our site.

So my question is, will Google revisit these pages based on their own records (as in, this page is indexed, let's go check it out again!), or will they only revisit them by following along a current site structure?

We are signed up with WMT if that helps.