How to safely exclude search result pages from Google's index?
-
Hello everyone,
I'm wondering what's the best way to prevent/block search result pages from being indexed by Google. The way search works on my site is that search form generates URLs like:
/index.php?blah-blah-search-results-blahI wanted to block everything of that sort, but how do I do it without blocking /index.php ?
Thanks in advance and have a great day everyone!
-
Hi Louise,
If you can ID the parameters, you can also look at blocking these in Webmaster Tools. This page explains more. As with any blocking of URLs, of course, proceed with caution.
-
I agree that can be effective. The reason I suggested the robots.txt is because Louise mentioned "blocking and preventing" as an objective. Robots.txt are particularly useful in the example where results from a search bar or something of that nature is involved. A NOINDEX, FOLLOW will not prevent bots from getting tired and dizzy, whereas the robots.txt can "block and prevent" bots from crawling certain parameters.
With all of that said, I think it is important to understand whether you need the bots to crawl and not index (in which case Spencer's answer is correct), or if you need to prevent bots from crawling the parameters altogether.
Hope that is more clear
-
I'm not sure that robots.txt is effective when url parameters are involved.
I would just add a meta robots tag to the head section of the search results template:
-
If you are able to identify a url parameter, you may excluded them using robots.txt. Here is a great resource on Robots.txt - http://moz.com/learn/seo/robotstxt
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Page Performance
Not long ago, I had a couple of peers asking why I was using sessions to evaluate page performance. They said it wasn't a good metric for evaluating a single page because it only looked at how many site visitors began their journey through you site form that page. They were trying to convert me over to pageviews, which they said was a superior metric because it show you every time that page had been loaded and therefore provided better insight. Moz uses sessions on their landing page report. Is this because it's an SEO tool, so all they are concerned with is how individual URLs attract site traffic? Signed, Confused in California
Reporting & Analytics | | PGD20110 -
Google Analytics global page empty
Hello, I don't know what happened with my Google Analytics initial page. For around a week I have there all my websites, but they show no results on global page. If I enter on any property I can see everything as normal, just not on the initial page. Screenshot: http://imgur.com/JKFVIh4 For sure I changed anything that I shouldn't. Perhaps you know what's wrong? Thank you in advance.
Reporting & Analytics | | prozis0 -
Google analytics question
Ok so in my traffic sources break down I have 3 sections: direct, organic, and referral. My question is under the referral tab I have recently noticed a new traffic source, my own website.... How is this possible? My top referring site is my own website.... Is this considered direct traffic or how is this being traced?
Reporting & Analytics | | jameswalkerson0 -
Google Analytics-Unique visitors?
Does organic search data show unique visitors or all visits? For example, if someone ( a single person) visits my site multiple times after searching the same keyword phrase, does that show in my analytics data as 1 hit from that keyword, or all hits? Thanks!
Reporting & Analytics | | RickyShockley0 -
Tracking an onpage 'event'
Hi all Wondering if anyone could help out with this one please. My client is a government backed free internet safety website and in the next few days they will be launching an update on all of their pages which will let people know if their browser is out of date. For example, when you go to their site you will get a message advising you to upgrade your browser for security reasons. They have employed the following code to check the browser The client are keen to know how many times in a period this message is shown to users. Any idea how one would go about tracking this please. Would it involve some custom GA work, would I be able to track the hits on https://wxxxx.org/javascript/update.js in GA? I'm a little stumped. Obviously I can tell how many people loaded the page but not sure how to work out what % of them see the javascript Many thanks for your help Carl
Reporting & Analytics | | GrumpyCarl0 -
(Not provided) organic search results in Analytics
Hi, So from what I've read, (not provided) organic search results in Analytics is a result of the user being logged into Google. The problem I have is about 20% of the organic search results are (Not provided), so what is mainly left are branded search terms. I think I'm clutching at straws, but is there anything I can do to see these organic traffic sources?
Reporting & Analytics | | JuiceBoxOM0 -
Does anyone know what's happened to google analytics -> traffic sources -> SEO -> queries many of my accounts are showing a drop to zero in the laste few days
Howdy mozzers It's in the question title really. Zero impressions showing for the last few days on. Multiple accounts Any thought out there
Reporting & Analytics | | Big_Partnership0 -
Tracking page visits
Hi guys After a lot of effort and late nights our website has gone online. http://www.health2000.co.nz/ In google Analytics am I able to see what pages individuals clicked on before they went to the cart? Ive had a look at GA 'Flow Visualsiton' but it seems a little confusing. Thanks for your help Pete
Reporting & Analytics | | dawsonski0