undefined
Skip to content
Moz logo Menu open Menu close
  • Products
    • Moz Pro
    • Moz Pro Home
    • Moz Local
    • Moz Local Home
    • STAT
    • Moz API
    • Moz API Home
    • Compare SEO Products
    • Moz Data
  • Free SEO Tools
    • Domain Analysis
    • Keyword Explorer
    • Link Explorer
    • Competitive Research
    • MozBar
    • More Free SEO Tools
  • Learn SEO
    • Beginner's Guide to SEO
    • SEO Learning Center
    • Moz Academy
    • SEO Q&A
    • Webinars, Whitepapers, & Guides
  • Blog
  • Why Moz
    • Agency Solutions
    • Enterprise Solutions
    • Small Business Solutions
    • Case Studies
    • The Moz Story
    • New Releases
  • Log in
  • Log out
  • Products
    • Moz Pro

      Your all-in-one suite of SEO essentials.

    • Moz Local

      Raise your local SEO visibility with complete local SEO management.

    • STAT

      SERP tracking and analytics for enterprise SEO experts.

    • Moz API

      Power your SEO with our index of over 44 trillion links.

    • Compare SEO Products

      See which Moz SEO solution best meets your business needs.

    • Moz Data

      Power your SEO strategy & AI models with custom data solutions.

    NEW Keyword Suggestions by Topic
    Moz Pro

    NEW Keyword Suggestions by Topic

    Learn more
  • Free SEO Tools
    • Domain Analysis

      Get top competitive SEO metrics like DA, top pages and more.

    • Keyword Explorer

      Find traffic-driving keywords with our 1.25 billion+ keyword index.

    • Link Explorer

      Explore over 40 trillion links for powerful backlink data.

    • Competitive Research

      Uncover valuable insights on your organic search competitors.

    • MozBar

      See top SEO metrics for free as you browse the web.

    • More Free SEO Tools

      Explore all the free SEO tools Moz has to offer.

    NEW Keyword Suggestions by Topic
    Moz Pro

    NEW Keyword Suggestions by Topic

    Learn more
  • Learn SEO
    • Beginner's Guide to SEO

      The #1 most popular introduction to SEO, trusted by millions.

    • SEO Learning Center

      Broaden your knowledge with SEO resources for all skill levels.

    • On-Demand Webinars

      Learn modern SEO best practices from industry experts.

    • How-To Guides

      Step-by-step guides to search success from the authority on SEO.

    • Moz Academy

      Upskill and get certified with on-demand courses & certifications.

    • MozCon

      Save on Early Bird tickets and join us in London or New York City

    Unlock flexible pricing & new endpoints
    Moz API

    Unlock flexible pricing & new endpoints

    Find your plan
  • Blog
  • Why Moz
    • Small Business Solutions

      Uncover insights to make smarter marketing decisions in less time.

    • Agency Solutions

      Earn & keep valuable clients with unparalleled data & insights.

    • Enterprise Solutions

      Gain a competitive edge in the ever-changing world of search.

    • The Moz Story

      Moz was the first & remains the most trusted SEO company.

    • Case Studies

      Explore how Moz drives ROI with a proven track record of success.

    • New Releases

      Get the scoop on the latest and greatest from Moz.

    Surface actionable competitive intel
    New Feature

    Surface actionable competitive intel

    Learn More
  • Log in
    • Moz Pro
    • Moz Local
    • Moz Local Dashboard
    • Moz API
    • Moz API Dashboard
    • Moz Academy
  • Avatar
    • Moz Home
    • Notifications
    • Account & Billing
    • Manage Users
    • Community Profile
    • My Q&A
    • My Videos
    • Log Out

The Moz Q&A Forum

  • Forum
  • Questions
  • Users
  • Ask the Community

Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

  1. Home
  2. SEO Tactics
  3. Technical SEO
  4. Blocking URL's with specific parameters from Googlebot

Moz Q&A is closed.

After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.

Blocking URL's with specific parameters from Googlebot

Technical SEO
2
10
3.8k
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as question
Log in to reply
This topic has been deleted. Only users with question management privileges can see it.
  • aethereal
    aethereal last edited by Oct 3, 2011, 7:30 AM

    Hi,

    I've discovered that Googlebot's are voting on products listed on our website and as a result are creating negative ratings by placing votes from 1 to 5 for every product. The voting function is handled using Javascript, as shown below, and the script prevents multiple votes so most products end up with a vote of 1, which translates to "poor".

    How do I go about using robots.txt to block a URL with specific parameters only? I'm worried that I might end up blocking the whole product listing, which would result in de-listing from Google and the loss of many highly ranked pages.

    DON'T want to block:

    http://www.mysite.com/product.php?productid=1234

    WANT to block:

    http://www.mysite.com/product.php?mode=vote&productid=1234&vote=2

    Javacript button code:

    onclick="javascript: document.voteform.submit();"

    Thanks in advance for any advice given.

    Regards,
    Asim

    1 Reply Last reply Reply Quote 0
    • AlanMosley
      AlanMosley @aethereal last edited by Oct 3, 2011, 11:57 PM Oct 3, 2011, 11:57 PM

      Good to hear, I am glad you perservered

      1 Reply Last reply Reply Quote 1
      • aethereal
        aethereal @aethereal last edited by Oct 3, 2011, 9:44 AM Oct 3, 2011, 9:44 AM

        Tried them all now and all come back with "Success"... May be I'll post in the WMT Forum and see if anyone can shed light on this problem. Thanks for your help Alan, it's much appreciated.

        1 Reply Last reply Reply Quote 0
        • AlanMosley
          AlanMosley @aethereal last edited by Oct 3, 2011, 9:16 AM Oct 3, 2011, 9:16 AM

          Yes correct, did you try the other formats?

          1 Reply Last reply Reply Quote 0
          • aethereal
            aethereal @aethereal last edited by Oct 3, 2011, 9:07 AM Oct 3, 2011, 9:07 AM

            Tried "Fetch as Googlebot" in Diagnostics and it came back as "Success" so I guess the robots.txt directive is not working. I'm assuming it should have reported a failure message when attempting to fetch a URL containing "?mode=vote".

            1 Reply Last reply Reply Quote 0
            • AlanMosley
              AlanMosley @aethereal last edited by Oct 3, 2011, 8:51 AM Oct 3, 2011, 8:51 AM

              Wrong place, go to diagnostics, then look for fetch as googlebot

              1 Reply Last reply Reply Quote 0
              • aethereal
                aethereal @aethereal last edited by Oct 3, 2011, 8:44 AM Oct 3, 2011, 8:44 AM

                I added "Disallow: /mode=vote" to the robots.txt file and also manually entered it on Crawler Access page, then clicked "Test" and no errors were reported. The WMT page states that robots.txt was last downloaded 16 hours ago so I'll wait until it picks the file up again and then check for any errors. Hopefully that will do trick 🙂

                1 Reply Last reply Reply Quote 0
                • AlanMosley
                  AlanMosley @aethereal last edited by Oct 3, 2011, 8:26 AM Oct 3, 2011, 8:24 AM

                  Try this in robots.txt, I did not think that Google allows wild cards but i just read that they do.

                  Disallow: /*mode=vote*

                  or

                  Disallow: /*mode=vote

                  or

                  Disallow: /*mode

                  Then try in Google WMT to read with googlebot to see if it works.

                  The first in the list seems right to me, but I have seen others do it the other ways.

                  1 Reply Last reply Reply Quote 1
                  • aethereal
                    aethereal @AlanMosley last edited by Oct 3, 2011, 8:18 AM Oct 3, 2011, 8:18 AM

                    Thanks for the reply. The site was developed using PHP, mySQL and Javascript. I was hoping there was a way to do it without getting programmers involved...

                    AlanMosley aethereal 7 Replies Last reply Oct 3, 2011, 11:57 PM Reply Quote 0
                    • AlanMosley
                      AlanMosley last edited by Oct 3, 2011, 7:53 AM Oct 3, 2011, 7:53 AM

                      dont think you are going to do it in robots.txt, rather do a 301 from mode=vote to non mode vote.

                      If you dont know how to put this into practise, tell me what your site is built with, if it is ASP.NET, i will show you how to impliment, if not someone else should be able to help.

                      aethereal 1 Reply Last reply Oct 3, 2011, 8:18 AM Reply Quote 0
                      • 1 / 1
                      1 out of 10
                      • First post
                        1/10
                        Last post

                      Got a burning SEO question?

                      Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.


                      Start my free trial


                      Browse Questions

                      Explore more categories

                      • Moz Tools

                        Chat with the community about the Moz tools.

                      • SEO Tactics

                        Discuss the SEO process with fellow marketers

                      • Community

                        Discuss industry events, jobs, and news!

                      • Digital Marketing

                        Chat about tactics outside of SEO

                      • Research & Trends

                        Dive into research and trends in the search industry.

                      • Support

                        Connect on product support and feature requests.

                      • See all categories

                      Related Questions

                      • ugorayan

                        My Website's Home Page is Missing on Google SERP

                        Hi All, I have a WordPress website which has about 10-12 pages in total. When I search for the brand name on Google Search, the home page URL isn't appearing on the result pages while the rest of the pages are appearing. There're no issues with the canonicalization or meta titles/descriptions as such. What could possibly the reason behind this aberration? Looking forward to your advice! Cheers

                        Technical SEO | Aug 1, 2018, 2:54 PM | ugorayan
                        0
                      • RodneyRiley

                        What's the best way to handle product filter URLs?

                        I've been researching and can't find a clear cut answer. Imagine you have a product category page e.g. domain/jeans You've a lot of options as to how to filter the results domain/jeans?=ladies,skinny,pink,10 or domain/jeans/ladies-skinny-pink-10 or domain/jeans/ladies/skinny?=pink,10 And in this how do you handle titles, breadcrumbs etc. Is the a way you prefer to handle filters and why do you do it that way? I'm trying to make my mind up as some very big names handle this differently e.g. http://www.next.co.uk/shop/gender-women-category-jeans/colour-pink-fit-skinny-size-10r VS https://www.matalan.co.uk/womens/shop-by-category/jeans?utf8=✓&[facet_filter][meta.tertiary_category][Skinny]=on&[facet_filter][variants.meta.size][Size+10]=on&[facet_filter][meta.master_colour][Midwash]=on&[facet_filter][min_current_price][gte]=6.0&[facet_filter][min_current_price][lte]=18.0&per=36&sort=

                        Technical SEO | Jul 12, 2017, 10:55 AM | RodneyRiley
                        0
                      • brianvest

                        Absolute URL or Relative URL in my sitemap?

                        It there a benefit to using and absolute url in my sitemap vs relative?

                        Technical SEO | Feb 10, 2016, 5:18 PM | brianvest
                        0
                      • ey_sja

                        How Does Dynamic Content for a Specific URL Impact SEO?

                        Example URL:  http://www.sja.ca/English/Community-Services/Pages/Therapy Dog Services/default.aspx The above page is generated dynamically depending on what province the visitor visits from.  For example, a visitor from BC would see something quite different than a visitor from Nova Scotia; the intent is that the information shown should be relevant to the user of that province. How does this effect SEO?  How (or from what location) does Googlebot decide to crawl the page? I have considered a subdirectory for each province, though that comes with its challenges as well.  One such challenge is duplicate content when different provinces may have the same information for some pages.  Any suggestions for this?

                        Technical SEO | Nov 17, 2014, 5:11 PM | ey_sja
                        0
                      • RoxBrock

                        Is it good practice to still pay for Best of the Web Directory (BOTW) and other similar one's you have to pay for?

                        I know that paid for links are hit by Google, but in the past these directories were okay. What about now? Thank you.

                        Technical SEO | Oct 28, 2015, 12:33 PM | RoxBrock
                        0
                      • inlinear

                        Are Collapsible DIV's SEO-Friendly?

                        When I have a long article about a single topic with sub-topics I can make it user friendlier when I limit the text and hide text just showing the next headlines, by using expandable-collapsible div's. My doubt is if Google is really able to read onclick textlinks (with javaScript) or if it could be "seen" as hidden text? I think I read in the SEOmoz Users Guide, that all javaScript "manipulated" contend will not be crawled. So from SEOmoz's Point of View I should better make use of old school named anchors and a side-navigation to jump to the sub-topics? (I had a similar question in my post before, but I did not use the perfect terms to describe what I really wanted. Also my text is not too long (<1000 Words) that I should use pagination with rel="next" and rel="prev" attributes.) THANKS for every answer 🙂

                        Technical SEO | Feb 12, 2013, 11:58 PM | inlinear
                        0
                      • joony2008

                        Found a Typo in URL, what's the best practice to fix it?

                        Wordpress 3.4, Yoast, Multisite The URL is supposed to be "www.myexample.com/great-site" but I just found that it's "www.myexample.com/gre-atsite" It is a relatively new site but we already pointed several internal links to "www.myexample.com/gre-atsite" What's the best practice to correct this? Which option is more desirable? 1.Creating a new page I found that Yoast has "301 redirect" option in the Advanced tap Can I just create a new page(exact same page) and put noindex, nofollow and redirect it to http://www.myexample.com/great-site OR 2. htacess redirect rule simply change the URL to http://www.myexample.com/great-site and update it, and add Options +FollowSymLinks RewriteEngine On
                        RewriteCond %{HTTP_HOST} ^http://www.myexample.com/gre-atsite$ [NC]
                        RewriteRule ^(.*)$ http://www.myexample.com/great-site$1 [R=301,L]

                        Technical SEO | Jun 19, 2012, 12:53 AM | joony2008
                        0
                      • wparlaman

                        What's the difference between a category page and a content page

                        Hello, Little confused on this matter. From a website architectural and content stand point, what is the difference between a category page and a content page? So lets say I was going to build a website around tea. My home page would be about tea. My category pages would be: White Tea, Black Tea, Oolong Team and British Tea correct? (  I Would write content for each of these topics on their respective category pages correct?) Then suppose I wrote articles on organic white tea, white tea recipes, how to brew white team etc...( Are these content pages?) Do I think link FROM my category page ( White Tea) to my ( Content pages ie; Organic White Tea, white tea receipes etc) or do I link from my content page to my category page? I hope this makes sense. Thanks, Bill

                        Technical SEO | May 22, 2011, 5:03 PM | wparlaman
                        0

                      Get started with Moz Pro!

                      Unlock the power of advanced SEO tools and data-driven insights.

                      Start my free trial
                      Products
                      • Moz Pro
                      • Moz Local
                      • Moz API
                      • Moz Data
                      • STAT
                      • Product Updates
                      Moz Solutions
                      • SMB Solutions
                      • Agency Solutions
                      • Enterprise Solutions
                      Free SEO Tools
                      • Domain Authority Checker
                      • Link Explorer
                      • Keyword Explorer
                      • Competitive Research
                      • Brand Authority Checker
                      • Local Citation Checker
                      • MozBar Extension
                      • MozCast
                      Resources
                      • Blog
                      • SEO Learning Center
                      • Help Hub
                      • Beginner's Guide to SEO
                      • How-to Guides
                      • Moz Academy
                      • API Docs
                      About Moz
                      • About
                      • Team
                      • Careers
                      • Contact
                      Why Moz
                      • Case Studies
                      • Testimonials
                      Get Involved
                      • Become an Affiliate
                      • MozCon
                      • Webinars
                      • Practical Marketer Series
                      • MozPod
                      Connect with us

                      Contact the Help team

                      Join our newsletter
                      Moz logo
                      © 2021 - 2025 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
                      • Accessibility
                      • Terms of Use
                      • Privacy

                      Looks like your connection to Moz was lost, please wait while we try to reconnect.