undefined
Skip to content
Moz logo Menu open Menu close
  • Products
    • Moz Pro
    • Moz Pro Home
    • Moz Local
    • Moz Local Home
    • STAT
    • Moz API
    • Moz API Home
    • Compare SEO Products
    • Moz Data
  • Free SEO Tools
    • Domain Analysis
    • Keyword Explorer
    • Link Explorer
    • Competitive Research
    • MozBar
    • More Free SEO Tools
  • Learn SEO
    • Beginner's Guide to SEO
    • SEO Learning Center
    • Moz Academy
    • SEO Q&A
    • Webinars, Whitepapers, & Guides
  • Blog
  • Why Moz
    • Agency Solutions
    • Enterprise Solutions
    • Small Business Solutions
    • Case Studies
    • The Moz Story
    • New Releases
  • Log in
  • Log out
  • Products
    • Moz Pro

      Your all-in-one suite of SEO essentials.

    • Moz Local

      Raise your local SEO visibility with complete local SEO management.

    • STAT

      SERP tracking and analytics for enterprise SEO experts.

    • Moz API

      Power your SEO with our index of over 44 trillion links.

    • Compare SEO Products

      See which Moz SEO solution best meets your business needs.

    • Moz Data

      Power your SEO strategy & AI models with custom data solutions.

    NEW Keyword Suggestions by Topic
    Moz Pro

    NEW Keyword Suggestions by Topic

    Learn more
  • Free SEO Tools
    • Domain Analysis

      Get top competitive SEO metrics like DA, top pages and more.

    • Keyword Explorer

      Find traffic-driving keywords with our 1.25 billion+ keyword index.

    • Link Explorer

      Explore over 40 trillion links for powerful backlink data.

    • Competitive Research

      Uncover valuable insights on your organic search competitors.

    • MozBar

      See top SEO metrics for free as you browse the web.

    • More Free SEO Tools

      Explore all the free SEO tools Moz has to offer.

    NEW Keyword Suggestions by Topic
    Moz Pro

    NEW Keyword Suggestions by Topic

    Learn more
  • Learn SEO
    • Beginner's Guide to SEO

      The #1 most popular introduction to SEO, trusted by millions.

    • SEO Learning Center

      Broaden your knowledge with SEO resources for all skill levels.

    • On-Demand Webinars

      Learn modern SEO best practices from industry experts.

    • How-To Guides

      Step-by-step guides to search success from the authority on SEO.

    • Moz Academy

      Upskill and get certified with on-demand courses & certifications.

    • MozCon

      Save on Early Bird tickets and join us in London or New York City

    Unlock flexible pricing & new endpoints
    Moz API

    Unlock flexible pricing & new endpoints

    Find your plan
  • Blog
  • Why Moz
    • Small Business Solutions

      Uncover insights to make smarter marketing decisions in less time.

    • Agency Solutions

      Earn & keep valuable clients with unparalleled data & insights.

    • Enterprise Solutions

      Gain a competitive edge in the ever-changing world of search.

    • The Moz Story

      Moz was the first & remains the most trusted SEO company.

    • Case Studies

      Explore how Moz drives ROI with a proven track record of success.

    • New Releases

      Get the scoop on the latest and greatest from Moz.

    Surface actionable competitive intel
    New Feature

    Surface actionable competitive intel

    Learn More
  • Log in
    • Moz Pro
    • Moz Local
    • Moz Local Dashboard
    • Moz API
    • Moz API Dashboard
    • Moz Academy
  • Avatar
    • Moz Home
    • Notifications
    • Account & Billing
    • Manage Users
    • Community Profile
    • My Q&A
    • My Videos
    • Log Out

The Moz Q&A Forum

  • Forum
  • Questions
  • Users
  • Ask the Community

Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

  1. Home
  2. SEO Tactics
  3. Technical SEO
  4. Blocking URL's with specific parameters from Googlebot

Moz Q&A is closed.

After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.

Blocking URL's with specific parameters from Googlebot

Technical SEO
2
10
3.8k
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as question
Log in to reply
This topic has been deleted. Only users with question management privileges can see it.
  • aethereal
    aethereal last edited by Oct 3, 2011, 7:30 AM

    Hi,

    I've discovered that Googlebot's are voting on products listed on our website and as a result are creating negative ratings by placing votes from 1 to 5 for every product. The voting function is handled using Javascript, as shown below, and the script prevents multiple votes so most products end up with a vote of 1, which translates to "poor".

    How do I go about using robots.txt to block a URL with specific parameters only? I'm worried that I might end up blocking the whole product listing, which would result in de-listing from Google and the loss of many highly ranked pages.

    DON'T want to block:

    http://www.mysite.com/product.php?productid=1234

    WANT to block:

    http://www.mysite.com/product.php?mode=vote&productid=1234&vote=2

    Javacript button code:

    onclick="javascript: document.voteform.submit();"

    Thanks in advance for any advice given.

    Regards,
    Asim

    1 Reply Last reply Reply Quote 0
    • AlanMosley
      AlanMosley @aethereal last edited by Oct 3, 2011, 11:57 PM Oct 3, 2011, 11:57 PM

      Good to hear, I am glad you perservered

      1 Reply Last reply Reply Quote 1
      • aethereal
        aethereal @aethereal last edited by Oct 3, 2011, 9:44 AM Oct 3, 2011, 9:44 AM

        Tried them all now and all come back with "Success"... May be I'll post in the WMT Forum and see if anyone can shed light on this problem. Thanks for your help Alan, it's much appreciated.

        1 Reply Last reply Reply Quote 0
        • AlanMosley
          AlanMosley @aethereal last edited by Oct 3, 2011, 9:16 AM Oct 3, 2011, 9:16 AM

          Yes correct, did you try the other formats?

          1 Reply Last reply Reply Quote 0
          • aethereal
            aethereal @aethereal last edited by Oct 3, 2011, 9:07 AM Oct 3, 2011, 9:07 AM

            Tried "Fetch as Googlebot" in Diagnostics and it came back as "Success" so I guess the robots.txt directive is not working. I'm assuming it should have reported a failure message when attempting to fetch a URL containing "?mode=vote".

            1 Reply Last reply Reply Quote 0
            • AlanMosley
              AlanMosley @aethereal last edited by Oct 3, 2011, 8:51 AM Oct 3, 2011, 8:51 AM

              Wrong place, go to diagnostics, then look for fetch as googlebot

              1 Reply Last reply Reply Quote 0
              • aethereal
                aethereal @aethereal last edited by Oct 3, 2011, 8:44 AM Oct 3, 2011, 8:44 AM

                I added "Disallow: /mode=vote" to the robots.txt file and also manually entered it on Crawler Access page, then clicked "Test" and no errors were reported. The WMT page states that robots.txt was last downloaded 16 hours ago so I'll wait until it picks the file up again and then check for any errors. Hopefully that will do trick 🙂

                1 Reply Last reply Reply Quote 0
                • AlanMosley
                  AlanMosley @aethereal last edited by Oct 3, 2011, 8:26 AM Oct 3, 2011, 8:24 AM

                  Try this in robots.txt, I did not think that Google allows wild cards but i just read that they do.

                  Disallow: /*mode=vote*

                  or

                  Disallow: /*mode=vote

                  or

                  Disallow: /*mode

                  Then try in Google WMT to read with googlebot to see if it works.

                  The first in the list seems right to me, but I have seen others do it the other ways.

                  1 Reply Last reply Reply Quote 1
                  • aethereal
                    aethereal @AlanMosley last edited by Oct 3, 2011, 8:18 AM Oct 3, 2011, 8:18 AM

                    Thanks for the reply. The site was developed using PHP, mySQL and Javascript. I was hoping there was a way to do it without getting programmers involved...

                    AlanMosley aethereal 7 Replies Last reply Oct 3, 2011, 11:57 PM Reply Quote 0
                    • AlanMosley
                      AlanMosley last edited by Oct 3, 2011, 7:53 AM Oct 3, 2011, 7:53 AM

                      dont think you are going to do it in robots.txt, rather do a 301 from mode=vote to non mode vote.

                      If you dont know how to put this into practise, tell me what your site is built with, if it is ASP.NET, i will show you how to impliment, if not someone else should be able to help.

                      aethereal 1 Reply Last reply Oct 3, 2011, 8:18 AM Reply Quote 0
                      • 1 / 1
                      1 out of 10
                      • First post
                        1/10
                        Last post

                      Got a burning SEO question?

                      Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.


                      Start my free trial


                      Browse Questions

                      Explore more categories

                      • Moz Tools

                        Chat with the community about the Moz tools.

                      • SEO Tactics

                        Discuss the SEO process with fellow marketers

                      • Community

                        Discuss industry events, jobs, and news!

                      • Digital Marketing

                        Chat about tactics outside of SEO

                      • Research & Trends

                        Dive into research and trends in the search industry.

                      • Support

                        Connect on product support and feature requests.

                      • See all categories

                      Related Questions

                      • Smileworks_Liverpool

                        What's the best way for users to upload their images to my wordpress site to promote UGC

                        I have looked at lots of different plugins and wanted a recommendation for an easy way for patients of ours to upload pictures of them out partying and having fun and looking beautiful so future users can see the final results instead of sometimes gory or difficult to understand before and after images. I'd like to give them the opportunity to write captions (like facebook or insta posts and would offer them incentives to do so. I don't want it to be too complicated for them or have too many steps or barriers but I do want it to look nice and slick and modern. Also do you think this would have a positive impact on SEO? I was also thinking of a Q&A app where dentists could get Q&A emails and respond - i've been doing AMA sessions and they've been really successful and I would like to bring it into out site and make it native. Thanks in advance 🙂

                        Technical SEO | Mar 14, 2023, 1:25 PM | Smileworks_Liverpool
                        1
                      • MTalhaImtiaz

                        Are image pages considered 'thin' content pages?

                        I am currently doing a site audit. The total number of pages on the website are around 400... 187 of them are image pages and coming up as 'zero' word count in Screaming Frog report. I needed to know if they will be considered 'thin' content by search engines? Should I include them as an issue? An answer would be most appreciated.

                        Technical SEO | Dec 6, 2014, 3:21 PM | MTalhaImtiaz
                        0
                      • catalinmoraru

                        Blocked URL parameters can still be crawled and indexed by google?

                        Hy guys, I have two questions and one might be a dumb question but there it goes. I just want to be sure that I understand: IF I tell webmaster tools to ignore an URL Parameter, will google still index and rank my url? IS it ok if I don't append in the url structure the brand filter?, will I still rank for that brand? Thanks, PS: ok 3 questions :)...

                        Technical SEO | Jan 29, 2015, 2:23 PM | catalinmoraru
                        0
                      • jfmonfette

                        XML Sitemap and unwanted URL parameters

                        We currently don't have an XML sitemap for our site. I generated one using Screaming Frog and it looks ok, but it also contains my tracking url parameters (ref=), which I don't want Google to use, as specified in GWT. Cleaning it will require time and effort which I currently don't have. I also think that having one could help us on Bing. So my question is: Is it better to submit a "so-so" sitemap than having none at all, or the risks are just too high? Could you explain what could go wrong? Thanks !

                        Technical SEO | Jan 20, 2014, 5:08 PM | jfmonfette
                        0
                      • BestRide

                        How to create site map for large site (ecommerce type) that has 1000's if not 100,000 of pages.

                        I know this is kind of a newbie question but I am having an amazing amount of trouble creating a sitemap for our site Bestride.com.  We just did a complete redesign (look and feel, functionality, the works) and now I am trying to create a site map.  Most of the generators I have used "break" after reaching some number of pages.  I am at a loss as to how to create the sitemap.  Any help would be greatly appreciated! Thanks

                        Technical SEO | Aug 12, 2013, 2:07 AM | BestRide
                        0
                      • ahockley

                        Google insists robots.txt is blocking... but it isn't.

                        I recently launched a new website.  During development, I'd enabled the option in WordPress to prevent search engines from indexing the site. When the site went public (over 24 hours ago), I cleared that option. At that point, I added a specific robots.txt file that only disallowed a couple directories of files.  You can view the robots.txt at http://photogeardeals.com/robots.txt Google (via Webmaster tools) is insisting that my robots.txt file contains a "Disallow: /" on line 2 and that it's preventing Google from indexing the site and preventing me from submitting a sitemap.  These errors are showing both in the sitemap section of Webmaster tools as well as the Blocked URLs section. Bing's webmaster tools are  able to read the site and sitemap just fine. Any idea why Google insists I'm disallowing everything even after telling it to re-fetch?

                        Technical SEO | Apr 2, 2013, 1:54 PM | ahockley
                        0
                      • CWseo

                        Why am I not showing up in the SERP's or Google Local?

                        I have been trying to optimise the following site for both Google SERP's and Google Local - Pixel Primate The URL has been around for around 3 years now but they just updated the website and launched it in December 2012. I did the on-page optimisation early in January 2013 and Google seems to have indexed the changes, for the home page at least. One major keyword I am targeting for the home page is 'Web Design Leicester'. I understand that the DA is fairly low (24) so this is something I need to improve. However, I've experienced positive results fairly quickly from just on-page optimisation for other sites I have worked on. The site just doesn't seem to be ranking at all for any keywords. Maybe the industry type is just extremely competitve but I find it very strange to not be visible anywhere in the SERPs. The site does not seem to have any penalties as it ranks for 'Pixel Primate' and all pages appear when doing a site: search. Also what's strange is that I set up the Google Local listing years ago but it doesn't appear anywhere in the local listing, not even when I search for it manually. Any suggestions would be appreciated.

                        Technical SEO | Jan 26, 2013, 6:02 PM | CWseo
                        0
                      • joony2008

                        Found a Typo in URL, what's the best practice to fix it?

                        Wordpress 3.4, Yoast, Multisite The URL is supposed to be "www.myexample.com/great-site" but I just found that it's "www.myexample.com/gre-atsite" It is a relatively new site but we already pointed several internal links to "www.myexample.com/gre-atsite" What's the best practice to correct this? Which option is more desirable? 1.Creating a new page I found that Yoast has "301 redirect" option in the Advanced tap Can I just create a new page(exact same page) and put noindex, nofollow and redirect it to http://www.myexample.com/great-site OR 2. htacess redirect rule simply change the URL to http://www.myexample.com/great-site and update it, and add Options +FollowSymLinks RewriteEngine On
                        RewriteCond %{HTTP_HOST} ^http://www.myexample.com/gre-atsite$ [NC]
                        RewriteRule ^(.*)$ http://www.myexample.com/great-site$1 [R=301,L]

                        Technical SEO | Jun 19, 2012, 12:53 AM | joony2008
                        0

                      Get started with Moz Pro!

                      Unlock the power of advanced SEO tools and data-driven insights.

                      Start my free trial
                      Products
                      • Moz Pro
                      • Moz Local
                      • Moz API
                      • Moz Data
                      • STAT
                      • Product Updates
                      Moz Solutions
                      • SMB Solutions
                      • Agency Solutions
                      • Enterprise Solutions
                      Free SEO Tools
                      • Domain Authority Checker
                      • Link Explorer
                      • Keyword Explorer
                      • Competitive Research
                      • Brand Authority Checker
                      • Local Citation Checker
                      • MozBar Extension
                      • MozCast
                      Resources
                      • Blog
                      • SEO Learning Center
                      • Help Hub
                      • Beginner's Guide to SEO
                      • How-to Guides
                      • Moz Academy
                      • API Docs
                      About Moz
                      • About
                      • Team
                      • Careers
                      • Contact
                      Why Moz
                      • Case Studies
                      • Testimonials
                      Get Involved
                      • Become an Affiliate
                      • MozCon
                      • Webinars
                      • Practical Marketer Series
                      • MozPod
                      Connect with us

                      Contact the Help team

                      Join our newsletter
                      Moz logo
                      © 2021 - 2025 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
                      • Accessibility
                      • Terms of Use
                      • Privacy

                      Looks like your connection to Moz was lost, please wait while we try to reconnect.