Googlebot found an extremely high number of URLs on your site:
-
Website: www.gobol.in
Although I have no indexed my search pages by adding /catalogsearch in robots.txt, still we are getting same error again and again
Here's a list of sample URLs with potential problems.
- http://www.gobol.in/catalogsearch/result/index/?category=&mobile_feature=4575_4578&q=panasonic+NR-BU303LH1H+REFRIGERATOR+296+L+GREY&special_price=32%2C456&x=0&y=0
- http://www.gobol.in/mobile-and-accessories/mobiles-and-brands.html?manufacturer=4753_3355_455_4435_4720_3407_2412_4728_4784_4790_2010_4789_4376_2469&operating_system_mobile=4612
Please help
-
Thanks Dirk
-
Hi Ravi,
Did you use the robots.txt tester in Webmastertools - your robots.txt mentions to block /catalogsearch - if you want to block the entire directory I think you have to use
Disallow: /catalogsearch/
On top of that /catalogsearch is only blocked for Googlebot-image - not for the "normal" Googlebot - if you want to have it blocked for all bots you have to move it to the beginning of the file. Idem for /admin => should be /admin/ and in the beginning of your robots.txt file
rgds,
Dirk
-
Thanks Martijn
-
Hi Ravi,
This message you got from Google is more a warning then an error, that is good to keep in mind. As there is no problem in theory with having a lot of URLs on your site if the quality is high enough. Therefore the first URL shouldn't cause any troubles, as like you mention, it's included in the robots.txt file.
The second URL is a bit less nice but shouldn't also be trouble, as it's making sure that the URL parameters are working. Also there is a canonical URL on all these filter pages to make sure the pages are originally linked back to the category page. It could be worth it to check the URL parameters within Google Webmaster Tools to make sure Google is aware of all the changes in content for different parameters.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
On site issues after Magento 2 launch
We did a new site launch on Feb 7th this year - www.vesternet.com It changed from Magento 1 to Magento 2. We had some launch issues around SEO but now we've solved most every crawler issue in Moz reporting - according to Moz we're in better shape on-site than ever. But our organic search is just dropping daily - we expected a drop after launch then back to normal, but over 2 months on something just isn't right. A good example, on Google UK for keyword 'home automation' we've always been about position 10, but now we're out the top 50... Forget about off site for now - what's wrong with our site itself to have caused this? Can anyone help with insights please as this is killing our sales
On-Page Optimization | | dbsmtec1 -
Company name in Site Meta description?
Ok. So I know that you should have your company name in the site title, but is it all that important in the site description? The reason I ask is because I am competing with another company for the #1 position (I was number 1, now #2) that has an 8 character name and mine is 22 taking away from a great deal of real estate in my 150-160 character site description in which I could provide additional information describing my company. Should I remove my name in the site description enabling me to use more descriptive keywords and actionable text such as (Find, research, contact, professional, info) etc. Thanks,
On-Page Optimization | | photoseo10 -
Phone number for SEO
We have had an interesting question from a client. They have asked if removing their phone number from their website would have an affect on their rankings. Our immediate answer was 'No' but it may affect the Brand, Usability and Customer experience by restricting the methods of contact. This then made us think that perhaps then it could have an effect in the long term. This situation is also complicated by the fact that they do not have a Google Local Plus account for operational, sensitivity reasons (they don't want to openly publicise their address) We believe that there shouldn't be any negative affect but thought we would open a discussion. Thanks in advance for any comments/ideas.
On-Page Optimization | | vital_hike0 -
How important are clean URLs?
Just wanting to understand the importance of clean URLs in regards to SEO effectiveness. Currently, we have URLs for a site that reads as follows: http://www.interhampers.com.au/c/90/Corporate Gift Hampers Should we look into modifying this so that the URL does not have % or figures?
On-Page Optimization | | Gavo1 -
Content with changing URL and duplicate content
Hi everyone, I have a question regarding content (user reviews), that are changing URL all the time. We get a lot of reviews from users that have been dining at our partner restaurants, which get posted on our site under (new) “reviews”. My worry however is that the URL for these reviews is changing all the time. The reason for this is that they start on page 1, and then get pushed down to page 2, and so on when new reviews come in. http://www.r2n.dk/restaurant-anmeldelser I’m guessing that this could cause for serious indexing problems? I can see in google that some reviews are indexed multiple times with different URLs, and some are not indexed at all. We further more have the specific reviews under each restaurant profile. I’m not sure if this could be considered duplicate content? Maybe we should tell google not to index the “new reviews section” by using robots.txt. We don’t get much traffic on these URLs anyways, and all reviews are still under each restaurant-profile. Or maybe the canonical tag can be used? I look forward to your input. Cheers, Christian
On-Page Optimization | | Christian_T2 -
Locating broken links on site?
Hey guys, I'm using Screaming Frog to help locate some broken links on a client's site and I've managed to pick up two. However, I can't seem to find whereabouts they're located on the site in order to fix them! Is there a way I can do this? Cheers!
On-Page Optimization | | Webrevolve0 -
What is this site all about?
Can anyone here tell me what this site is all about?... I really need to know, i have a blog.
On-Page Optimization | | Wales33030