How to safely exclude search result pages from Google's index?
-
Hello everyone,
I'm wondering what's the best way to prevent/block search result pages from being indexed by Google. The way search works on my site is that search form generates URLs like:
/index.php?blah-blah-search-results-blahI wanted to block everything of that sort, but how do I do it without blocking /index.php ?
Thanks in advance and have a great day everyone!
-
Hi Louise,
If you can ID the parameters, you can also look at blocking these in Webmaster Tools. This page explains more. As with any blocking of URLs, of course, proceed with caution.
-
I agree that can be effective. The reason I suggested the robots.txt is because Louise mentioned "blocking and preventing" as an objective. Robots.txt are particularly useful in the example where results from a search bar or something of that nature is involved. A NOINDEX, FOLLOW will not prevent bots from getting tired and dizzy, whereas the robots.txt can "block and prevent" bots from crawling certain parameters.
With all of that said, I think it is important to understand whether you need the bots to crawl and not index (in which case Spencer's answer is correct), or if you need to prevent bots from crawling the parameters altogether.
Hope that is more clear
-
I'm not sure that robots.txt is effective when url parameters are involved.
I would just add a meta robots tag to the head section of the search results template:
-
If you are able to identify a url parameter, you may excluded them using robots.txt. Here is a great resource on Robots.txt - http://moz.com/learn/seo/robotstxt
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How do I fix 608's please?
Hi, I'm on the free trial and finding it very useful I've fixed all my 301's. but now I have a load of 608's. I don't no what this is! I feel like I've cured herpes only to get gonorrhea! can any one help. I have 41 608's which is more than the 301's I had. I hope they are non-related! I won't bore you with the whole list but some of the url's are: Error Code 608: Page not Decodable as Specified Content Encoding http://sussexchef.com/catering-at-mr-mrs-currys-50th-wedding-anniversary/guestsarrive----608 Error Code 608: Page not Decodable as Specified Content Encoding http://sussexchef.com/funeral-catering/picture4-2----608 Error Code 608: Page not Decodable as Specified Content Encoding http://sussexchef.com/wedding-venues
Reporting & Analytics | | SussexChef831 -
Double click by Google - whats that all about?
Buongiorno from Latitude 53.92705600 Longitude -1.38481600... Ok ive watched the video, ive read the blog posts but what is Double click by Google? how do i get to use it?
Reporting & Analytics | | Nightwing
http://doubleclickadvertisers.blogspot.co.uk/2012/06/digital-marketing-platform-to-win.html Is it a seperate service to Adwords? Questions questions... Any insights welcome 🙂0 -
Google Tag Assistant for Chrome
I'm using the Google Tag Assistant for Chrome, and I noticed something really weird. No matter what pages I look at, the same two GA tags show up. It's weird. You can see the tag that is "working", and then there are two repeats. For example, when I look at this page, I see the GA tag that is working and then all the remarketing tags. Then I see UA-36732895-1 repeated twice. Anyone have any idea what this is? Thanks!
Reporting & Analytics | | PGD20110 -
No Query parameter for site search
Hi Guys, I have enable site search for analytics a number of times. But this time it's the first time I came across a search with no query parameters. example.com/search/item/searchterm What is the most simple way to approach this? thank you!
Reporting & Analytics | | GetApp0 -
Major practices which helps to index pages by google.
Actually, We have submitted more than 100 pages in to google through xml sitemap. But, we see in that 75% of the pages where indexed by google. Note : Excluding the duplicate pages
Reporting & Analytics | | Webworld_Norway0 -
How Google handles keywords and punctuation?
How does Google handle punctuation of keywords within its search? If I have three keywords together, but on the site there is a question mark between them, does Google ignore the question mark? Or should I think of a new way of writing the content so Google likes it more? And would this apply with articles (a, the, etc) and prepositions such as (in, of, etc.)? Thanks a lot.
Reporting & Analytics | | Rocket.Fuel1 -
Google Secure Search Announcement: How do you think this will play out?
So, I read this post on the Google Blog and then this commentary about it. Wondering what some of you veterans think about how this will impact results. I always remain logged out for queries and use several different software tools to check when I run an analysis. Since social influences are becoming a bigger factor, do you think this is going to create some issues in data? What are your thoughts?
Reporting & Analytics | | TheARKlady0 -
Google vs Bing/Yahoo SERP results?
I see large differences in Google and Bing/Yahoo SERP results for many keywords. Google shows many of our primary keywords in their top ten, while Bing/Yahoo rank the same one 40-50 or above. Do you have any insight on their differences that might account for this variation? We are legitimate, long time, white hatters at a small manufacturer that is one of the leaders in our industry. The only thing I can think of that might describe this issue is PPC advertising. Their algorithms wouldn't be that inconsistent would they? (We do much more paid advertising on Google) But generally this should have no affect, right? Thanks, George...
Reporting & Analytics | | rhawk0