Google Bot Noindex
-
If a site has the tag, can it still be flagged for duplicate content?
-
You can prevent Google from crawling the site through the robots.txt rather than the .htaccess Will.
-Andy
-
Cheers for the response Andy.
I do mean from a penalty point of view.
Can I block Google from crawling the site using the htaccess or robots.txt files?
I want the page to be indexed in all search engines but Google
I'm just experimenting and I don't mind whether the site with the duplicated content gets a penalty, but I would very much mind if the site whether the original content came from was given a penalty.
Will
-
Do you mean from a penalty point of view?
Google can still crawl and see what is on a page with a noindex flag, but it just won't index it. It doesn't guarantee that duplicate content won't still be an issue for you. This is one of those areas when it comes to Panda that people try. Some just noindex pages and have success with them, others need to remove them altogether.
-Andy
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Should I use the Google disavow tool?
Hi I'm a bit new to SEO and am looking for some guidance. Although there is no indication in Webmaster tools that my site is being penalised for bad links, I have noticed that I have over 200 spam links for "Pay Day Loans" pointing to my site. (This was due to a hack on my site several years ago). So my question is two fold. Firstly, is it normal to have spammy links pointing to your site and secondly, should I bother to do anything about it? I did some research into the Disavow tool in Webmaster tools wonder I should use it to block all these links. Thanks
Technical SEO | | hotchilidamo0 -
Why wont google Index this page?
A week ago i accidentally changed this page settings in my CMS to "disable & dont index" as i was going to replace this page with another, but this didnt happen, but i forgot to switch the settings back! http://www.over50choices.co.uk/funeral-planning/funeral-plans Anyhow in an effort to get it back up quickly i submitted in GWTs but its still not indexed. When i use several SEO on page checking tools it has the Meta Title data as "Form" and not the correct title. Any ideas please? Yours frustrated Ash
Technical SEO | | AshShep10 -
All images are noindex will opening this at once be an issue?
Hi, All images are noindex will opening this at once be an issue? Not sure how a few months ago all my images were set as noindex which i realized last week. We have 20K images which were indexed fine but now when i check Site:sitename it shows 10 or 12 and the inspect element via Chrome i see the noindex is set for all images. We have been renaming the images and adding ALT tags for most of them and would it be an issue if we change the noindex in one shot or should we do them few at a time? Thanks
Technical SEO | | mtthompsons0 -
Auto Generated Title by Google in SERP
Is there a way to tell Goolge to use my title instead of one generated by Google I have unique meta tags for each page, no duplicates, why Google is doing that? Thanks
Technical SEO | | KentR0 -
Mysterious drop of website ranking in google
Usually, I don't want to bother anybody by posting silly questions on forums. But this time I really might need advice. My wife and I took over the website maintenance and e-marketing of a local air conditioning company end of March this year. Before that the applied SEO strategies were not very user friendly and a little too search engine focused (spammy keyword stuffed articles, confusing website structure, a lot of directory links). Yesterday night (May 15th) the website more or less stopped ranking. For search terms like "ac repair englewood fl" or "trane north port" and many more the website was on page 1. Here are some more details: I replaced the old website with a newer version end of April. Since some of old the url structure did not apply any longer, I did a setup of around 30 301-redirects in .htaccess. The new site seemed to rank more or less as expected. The homepage has a PakeRank of 1 (seomoz Page Authority is 31). I am working on that but good natural links just take some time. site:kobiecomplete.com still brings up all the pages Google Webmaster Tools notified me on May 12th that there was a possible outage: _"_While crawling your site, we have noticed an increase in the number of transient soft 404 errors around 2012-05-08 16:00 UTC (London, Dublin, Edinburgh). Your site may have experienced outages. These issues may have been resolved. Here are some sample pages that resulted in soft 404 errors:" The listed pages under "some sample pages" are only pages from the old website which do not exist any longer and the 301 redirect was not setup. But this should have been already any issue before, if at all.
Technical SEO | | grojoh
I added the missing 301 redirects and marked them as fixed in Google Webmaster Tools. I had a copy of the website on a testing webspace (root directory of brightsidewg.com). Even though I had robots.txt set to disallow everything and WordPress search engine privacy set to do not index / follow, the website appeared on the Google search results yesterday night instead of the original website (kobiecomplete.com). Even though brightsidewg was a few ranks worse than kobiecomplete.com was, it was still ranking.
To remove the duplicate content, I deleted everything on brightsidewg.com and requested the removal of the website in the Webmaster Tools. Now brightsidewg.com is not any longer indexed (good) but it didn't help the ranking of kobiecomplete.com. Especially the homepage and the service area pages were ranking pretty decent on Google before yesterday night. Now I can not find them at all. Only other less important pages rank on page 8+ No malware on website I did not do any big changes on the website yesterday (only really minor ones). I did not acquire any weird/paid links even though there is a new link from a PageRank 0 website which I did not setup: http://www.indo-karya.com/detail/news/2012/kombise But that alone I think would not be enough for a penalty. It almost looks like that Google applied a partial -950 filter!? I could submit the website for reconsideration to Google and tell them about the duplicate content issue with my testing webspace brightsidewg.com. What do you think about it and what shall I do? Thank you so much for any help!0 -
Does Google Bot accept Cookies
I am working with a per page results refinement that stores a cookie on the users computer and then keeps that same per page as the user goes around the site. I was just wondering if that was true for Google bot or Bing bot as well. Will they keep the cookie or would they not be able to accept it. I just want to know as I dont want different urls created if they can keep the cookie. Thanks!
Technical SEO | | Gordian0 -
Why is google ignoring my sitelinks demotions?
I'm referring to the sitelinks that appear in the SERPs when searching for my brand name. 6 subpages come up- some found in my main navigation, some not. 3 of the 6 sitelinks have been demoted in Webmaster Tools under "Site Configuration > Sitelinks." I realize that they say in their instructions: "Google doesn't guarantee that demoted URLs will never appear as a sitelink, but we do consider a demotion a strong hint that we'll try to honor when generating sitelinks." I am just surprised they would ignore 3 of the demotions and for so long. The pages that are demoted do not have very many internal links pointing to them (unlike other pages of my site that are targeted specifically). Also, the site has tens of thousands of pages to choose from. Why are they ignoring my request? What else can I do to fix this?
Technical SEO | | Hakkasan0 -
.co what is it? Do I need it? Does Google hate it?
Do .com rank better then .co? I don't know much about .co so I'm just looking for some insight! Thanks in advance.
Technical SEO | | christinarule0