Googlebot found an extremely high number of URLs on your site:
-
Website: www.gobol.in
Although I have no indexed my search pages by adding /catalogsearch in robots.txt, still we are getting same error again and again
Here's a list of sample URLs with potential problems.
- http://www.gobol.in/catalogsearch/result/index/?category=&mobile_feature=4575_4578&q=panasonic+NR-BU303LH1H+REFRIGERATOR+296+L+GREY&special_price=32%2C456&x=0&y=0
- http://www.gobol.in/mobile-and-accessories/mobiles-and-brands.html?manufacturer=4753_3355_455_4435_4720_3407_2412_4728_4784_4790_2010_4789_4376_2469&operating_system_mobile=4612
Please help
-
Thanks Dirk
-
Hi Ravi,
Did you use the robots.txt tester in Webmastertools - your robots.txt mentions to block /catalogsearch - if you want to block the entire directory I think you have to use
Disallow: /catalogsearch/
On top of that /catalogsearch is only blocked for Googlebot-image - not for the "normal" Googlebot - if you want to have it blocked for all bots you have to move it to the beginning of the file. Idem for /admin => should be /admin/ and in the beginning of your robots.txt file
rgds,
Dirk
-
Thanks Martijn
-
Hi Ravi,
This message you got from Google is more a warning then an error, that is good to keep in mind. As there is no problem in theory with having a lot of URLs on your site if the quality is high enough. Therefore the first URL shouldn't cause any troubles, as like you mention, it's included in the robots.txt file.
The second URL is a bit less nice but shouldn't also be trouble, as it's making sure that the URL parameters are working. Also there is a canonical URL on all these filter pages to make sure the pages are originally linked back to the category page. It could be worth it to check the URL parameters within Google Webmaster Tools to make sure Google is aware of all the changes in content for different parameters.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
SEO for E-Commerce Sites
Hi Everybody, I have two e-commerce sites just launched with not much content at the moment just user login pages for the clients to avail the service. The management is not interested to put much content there i think. Maximum what they will be putting only 5 pages of content in total, not more than this. Any practical tips how to optimize such sites especially when there is not much content. Best
On-Page Optimization | | Sequelmed0 -
HTML Site SEO (NO CMS)
I have got a client site, which is dated (2007) and has not been shifted to any recognised CMS yet. It is HTML based. Is it possible to SEO on such a site? Is it even worth it? If it is possible to do SEO on this, any suggestions will be highly appreciated. Thank you.
On-Page Optimization | | ArthurRadtke3 -
URL Path. What is better for SEO
Hello Moz people, Is it better for SEO to have a URL path like this: flowersite.com/anniversary_flowers/dozen_roses OR flowersite.com/dozen_roses Is it better to have the full trail of pages in the URL?
On-Page Optimization | | CKerr0 -
Do https sites rank as well as http sites?
2 Questions: Question 1 - We currently have our entire site running on https (the http pages 301-redirect to the https versions). Assuming that the https pages load as quickly as the http versions, is it a problem that the entire site is https? The only official answer I've been able to find is this 2011 video where Matt Cutts basically says "I don't know" - http://www.youtube.com/watch?v=xeFo4ytOk8M Question 2 - Is there any problem with having half our site running on https only (with the http pages redirected), and the other half (our blog) running on http only (with all https blog pages redirected to the http versions)? Thanks in advance for any input! Justin
On-Page Optimization | | JustinClark0 -
"irrelevant pages of a site"
Hi there! Some pages of my site like "contact" or "registration": Should they have a title and a description tag? They are pages that I don't want them to be shown in the SERPs....Could I be penalized by google If I don't do so? The SEOMOZ crawling tool warms me about this issue (to short titles, no meta-description tags....) Many thanks
On-Page Optimization | | juanmiguelcr0 -
Title and Url Agreement
In the case of trying to hit a wide taxonomy, is it better to keep your title and URL in agreement, or to vary them slightly for exact search matching. For instance this blog post which has the following url: http://www.simplifiedbuilding.com/blog/build-your-own-standing-desk/ has the title "Make a Stand Up Desk - Better Working, Longer Living" The ideas is that build and make are similar words and "stand up" and "standing" are also similar. So what is the better way to go?
On-Page Optimization | | CPollock0 -
Why is our site dropping in Domain Authority
Hi all, I have a question. I have been working SEO on our site www.betxpert.com for the last 4-6 weeks and I have been looking forward to seeing the new DA and PA numbers. However today, our DA is now 30...where it was 31 or 33 a couple of days ago. How come the decrease? I have only improved the site i should say... Hope someone can shed a little light on the issue... -Rasmus
On-Page Optimization | | rasmusbang0