Googlebot found an extremely high number of URLs on your site:
-
Website: www.gobol.in
Although I have no indexed my search pages by adding /catalogsearch in robots.txt, still we are getting same error again and again
Here's a list of sample URLs with potential problems.
- http://www.gobol.in/catalogsearch/result/index/?category=&mobile_feature=4575_4578&q=panasonic+NR-BU303LH1H+REFRIGERATOR+296+L+GREY&special_price=32%2C456&x=0&y=0
- http://www.gobol.in/mobile-and-accessories/mobiles-and-brands.html?manufacturer=4753_3355_455_4435_4720_3407_2412_4728_4784_4790_2010_4789_4376_2469&operating_system_mobile=4612
Please help
-
Thanks Dirk
-
Hi Ravi,
Did you use the robots.txt tester in Webmastertools - your robots.txt mentions to block /catalogsearch - if you want to block the entire directory I think you have to use
Disallow: /catalogsearch/
On top of that /catalogsearch is only blocked for Googlebot-image - not for the "normal" Googlebot - if you want to have it blocked for all bots you have to move it to the beginning of the file. Idem for /admin => should be /admin/ and in the beginning of your robots.txt file
rgds,
Dirk
-
Thanks Martijn
-
Hi Ravi,
This message you got from Google is more a warning then an error, that is good to keep in mind. As there is no problem in theory with having a lot of URLs on your site if the quality is high enough. Therefore the first URL shouldn't cause any troubles, as like you mention, it's included in the robots.txt file.
The second URL is a bit less nice but shouldn't also be trouble, as it's making sure that the URL parameters are working. Also there is a canonical URL on all these filter pages to make sure the pages are originally linked back to the category page. It could be worth it to check the URL parameters within Google Webmaster Tools to make sure Google is aware of all the changes in content for different parameters.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
IP Canonicalization for HTTPS site?
I received an unsolicited SEO report for one of my sites. My site was faulted for not having IP canonicalization set up. I reviewed this carefully. My site runs on Apache, is https and is on a dedicated IP. The mod rewrite rules for Apache all deal with the http version of the site. When I type my site's IP into a browser, I get the the https version, but with a unsecure cert warning as the certificate does not include the IP. Should I implement the http IP canonicalization rule. Another rewrite rule would then redirect the request to the https version?
On-Page Optimization | | FatRodent20130 -
Backlink from High DA but low PA
Hi, Which back link has more value; 1. Backlink from High DA and low PA. 2. Backlink from High PA and Low DA Thanks
On-Page Optimization | | Kashif-Amin0 -
Changing a page url
I have a page that ranks well (#4) for a good keyword. However, the url has the keyword in it but is misspelled. I would like to change the url to have the correct spelling but do not want to lose the ranking that I have. What is the best and safest way to proceed?
On-Page Optimization | | bhsiao0 -
Language for URLs on new international websites
We are due to launch our new site and it will be targeting an international market. We have setup these new sites in the following way www.website.com/fr/content-goes-here www.website.com/es/content-goes-here This has been done in conjunction with setting up the parameters in GWT and making sure it is optimised for the language itself, and that countries search engine. But our web dev team have said that the URLs at the moment will be in english and not the native language, so if you were on the french version of our site you would see the url in english and not french. Will this negatively affect the site for SEO, and who else would think it would be negative from a usability perspective? Any help is appreciated.
On-Page Optimization | | RyanCrawf19840 -
URL contain special character
Hello, I am using URLs which contain special character such as ', ". I found in the Google Webmaster Tool report errors related URL contain ' character. Google have indexed partial URL from beginning to ' character and cut off the rest of URL. For example: I submitted URL www.example.com/vietnam-visa-corp'-test-page.html, then google report error NOT FOUND for URL www.example.com/vietnam-visa-corp I don't know why and how to fix it? Please help! Thanks,
On-Page Optimization | | JohnHuynh1 -
0 urls indexed in GWT, many found with site: command
Hi, This is happening with a brand new site. We have created sitemaps and submitted them to Google Webmaster Tools. GWT says sitemaps are ok, "x" number of urls submitted, but no urls indexed. When I check in Google with site:domain.com I see that many of the urls are already indexed. Why this discrepancy between GWT and reality? Thanks for your time!
On-Page Optimization | | gerardoH0 -
Replacing "_" with "-" in url, results in new url?
We ran SEOmoz's "On-Page Optimization" tool on a url which contains the character "_". According to the tool: "Characters which are less commonly used in URLs may cause problems with accessibility, interpretation and ranking in search engines. It is considered a best practice to stick to standard URL structures to avoid potential problems." "Rewrite the URL to contain only standard characters." Therefore we will rewrite the url, replacing "_" with "-". Will search engines consider the "-" url a different one? Do we need to 301 the old url to the new one? Thanks for your help!
On-Page Optimization | | gerardoH0 -
Site structure for services and blog articles
Hi, looking for some advice on the structure for a relatively small site (around 200 pages). I'd like a structure where we can talk about our services as well as write blog articles on topics that relate to our services. We'll have loads more content in the blog area than in the services area. I was thinking of this: option 1: /services /services/copywriting
On-Page Optimization | | JaspalX
/services/social-media
/services/press-releases etc. and categories for articles where we'd give tips, talk about trends etc. /copywriting
/social-media
/effective-press-releases
etc. would it be better to have a different structure, say: option 2: /copywriting
/copywriting/services
/copywriting/articles OR option 3: /copywriting-services
/copywriting-blog OR option 4: /services/copywriting
/blog/copywriting OR is there another, better way perhaps? Of course the internal anchor text links to the services/blog articles pages will be tuned to try and make it clear what each section is about i.e. our services vs. industry trends/comments/tips for the blog.0