How to stop Search Bot from crawling through a submit button

[NarenBansal]

On our website http://www.thefutureminders.com/, we have three form fields that have three pull downs for Month, Day, and year. This is creating duplicate pages while indexing. How do we tell the search Bot to index the page but not crawl through the submit button?

Thanks

Naren

[NarenBansal]

Hi Dan

What is happening is this - since we have all the months [12], all the dates [31] and years[1921 through 2011] in the form fields, the robot seems to be taking these incrementally and then using the submit button. After the submit button, user is presented with a registration page. While we do want the search to index the rest of the page and the crawl through the rest of the page links we do not want it to crawl through that submit button. I hope I am making sense.

Naren

[Hannahm24]

The advantage of blocking a page from being indexed via a meta tag is it is less likely to have unexpected consequences. I've often seen in the past cases where an incorrectly modified robots.txt file leads to a site being blocked by accident.

[evolvingSEO]

Hi

To my knowledge, you don't stop it from crawling through the button (like a nofollowed link), rather you block the robot at the page it ends up on after clicking submit.

Say the user hits submit and it takes them to mydomain.com/confirm.html  On that page you'll want to add;

....if you want it to NOT index the page but follow the links on it.

or

...if you want it to NOT index and NOT follow the links on that page.

Its advised that its better to do this with the meta tag than in robots.txt.

Hopefully I've understood the question correctly!

-Dan

[JamesNorquay]

Block the pages/folders you do not wish to be indexed with robots.txt file:

www.website.com/robots.txt

User-agent: *
Disallow: /folder1/
Disallow: /folder2/

OR you can add canonical tags to the other pages which are creating duplicate content.