Googlebot found an extremely high number of URLs on your site:
-
Website: www.gobol.in
Although I have no indexed my search pages by adding /catalogsearch in robots.txt, still we are getting same error again and again
Here's a list of sample URLs with potential problems.
- http://www.gobol.in/catalogsearch/result/index/?category=&mobile_feature=4575_4578&q=panasonic+NR-BU303LH1H+REFRIGERATOR+296+L+GREY&special_price=32%2C456&x=0&y=0
- http://www.gobol.in/mobile-and-accessories/mobiles-and-brands.html?manufacturer=4753_3355_455_4435_4720_3407_2412_4728_4784_4790_2010_4789_4376_2469&operating_system_mobile=4612
Please help
-
Thanks Dirk
-
Hi Ravi,
Did you use the robots.txt tester in Webmastertools - your robots.txt mentions to block /catalogsearch - if you want to block the entire directory I think you have to use
Disallow: /catalogsearch/
On top of that /catalogsearch is only blocked for Googlebot-image - not for the "normal" Googlebot - if you want to have it blocked for all bots you have to move it to the beginning of the file. Idem for /admin => should be /admin/ and in the beginning of your robots.txt file
rgds,
Dirk
-
Thanks Martijn
-
Hi Ravi,
This message you got from Google is more a warning then an error, that is good to keep in mind. As there is no problem in theory with having a lot of URLs on your site if the quality is high enough. Therefore the first URL shouldn't cause any troubles, as like you mention, it's included in the robots.txt file.
The second URL is a bit less nice but shouldn't also be trouble, as it's making sure that the URL parameters are working. Also there is a canonical URL on all these filter pages to make sure the pages are originally linked back to the category page. It could be worth it to check the URL parameters within Google Webmaster Tools to make sure Google is aware of all the changes in content for different parameters.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Does having a \ on a URL make a difference?
On our website we have CMS pages which end normally without a /. However when linking from web banners e.c.t it always adds a / on the end. Will this have a negative impact on SEO?
On-Page Optimization | | AHF_Furniture0 -
Consolidating a Large Site with Duplicate Content
I will be restructuring a large website for an OEM. They provide products & services for multiple industries, and the product/service offering is identical across all industries. I was looking at the site structure and ran a crawl test, and learned they have a LOT of duplicate content out there because of the way they set up their website. They have a page in the navigation for “solution”, aka what industry you are in. Once that is selected, you are taken to a landing page, and from there, given many options to explore products, read blogs, learn about the business, and contact them. The main navigation is removed. The URL structure is set up with folders, so no matter what you select after you go to your industry, the URL will be “domain.com/industry/next-page”. The product offerings, blogs available, and contact us pages do not vary by industry, so the content that can be found on “domain.com/industry-1/product-1” is identical to the content found on “domain.com/industry-2/product-1” and so-on and so-forth. This is a large site with a fair amount of traffic because it’s a pretty substantial OEM. Most of their content, however, is competing with itself because most of the pages on their website have duplicate content. I won’t begin my work until I can dive in to their GA and have more in-depth conversations with them about what kind of activity they’re tracking and why they set up the website this way. However, I don’t know how strategic they were in this set up and I don’t think they were aware that they had duplicate content. My first thought would be to work towards consolidating the way their site is set up, so we don’t spread the link-equity of “product-1” content, and direct all industries to one page, and track conversion paths a different way. However, I’ve never dealt with a site structure of this magnitude and don’t want to risk messing up their domain authority, missing redirect or URL mapping opportunities, or ruin the fact that their site is still performing well, even though multiple pages have the same content (most of which have high page authority and search visibility). I was curious if anyone has dealt with this before and if they have any recommendations for tackling something like this?
On-Page Optimization | | cassy_rich0 -
Changing a page url
I have a page that ranks well (#4) for a good keyword. However, the url has the keyword in it but is misspelled. I would like to change the url to have the correct spelling but do not want to lose the ranking that I have. What is the best and safest way to proceed?
On-Page Optimization | | bhsiao0 -
To change or not to change site URL structure?
I am learning my way around SEO, having always used professionals for it in the past on previous businesses i have decided to do it myself and learn more about it. Now the dilemma i am up against is i recently changed some of my permalinks on quite a few main pages throughout the site. The site launched in April this year so we're quite new. The problem is since my last change i have not seen any increase...a decrease which in fact hasn't recovered at all. Having now analysed them in more depth and read up more on the whole subject of SEO, (which is endless) i have put together a complete new strategy; with this increased understanding of what i am doing (but by no means conclusive) i want to complete a full overhaul on all SEO (via Wordpress which i use along with YOAST SEO tools), ensuring i have all my keywords, permalinks and descriptions spot on throughout every page, post and picture. I spent a lot of time mapping these out, ensuring there is no Focus keyword duplication, and that the site is relatively flat in terms of its layout. What i am unsure about now is whether changing my permalinks again is a bad thing to do?
On-Page Optimization | | MrCostello
Could it permanently damage my rep going forward?
Should i just focus on my content and keywords/descriptions? I am at a loss as i don't want to do irreparable damage to our reputation. The site is still reasonable easy to manage so changing now is the best time to do it, but if changing the URLs is a waste of time then i may just forget that and just work on the keywords, descriptions and content. Advice is 'oh so welcome' 🙂0 -
Do https sites rank as well as http sites?
2 Questions: Question 1 - We currently have our entire site running on https (the http pages 301-redirect to the https versions). Assuming that the https pages load as quickly as the http versions, is it a problem that the entire site is https? The only official answer I've been able to find is this 2011 video where Matt Cutts basically says "I don't know" - http://www.youtube.com/watch?v=xeFo4ytOk8M Question 2 - Is there any problem with having half our site running on https only (with the http pages redirected), and the other half (our blog) running on http only (with all https blog pages redirected to the http versions)? Thanks in advance for any input! Justin
On-Page Optimization | | JustinClark0 -
Sudden Site Rankings Drop
Good day guys, We have been following strict SEO strategies for the past 6 months, all sites have been improving incredibly well, all except one. The site in question is http://bit.ly/IH4pkM . The site is regarding automotive spray booth equipment. We were ranking on the first page for the keyword "spray booth" (which is the most important one), at place #4 for weeks on end. However since half-way last week, the site has been dropped to half-way the second page (#17). There are barely any crawler errors listed for our campaign on SEOMoz. There were several pages of which the meta description was missing, but that has been fixed earlier this week. When it comes to link building, I looked at what the top competitors were doing, and was looking for unique link building opportunities myself. We have received 0 webmaster tools warnings as well. I do not believe we are penalized due to the "penguin" update. After all, if you search for for the company's name in Google, it is still listed on there (# 2). Nor have we been part of dodgy link networks at all. So my question is, what do you guys believe made us drop the rankings? Is there some on-page issues I am overlooking? Any recommendations to restore out previous rankings? Kind Regards, Roderic
On-Page Optimization | | Michael-Goode0 -
URL Strucutre
Hi there, Need some advice please on URL structure. I have been doing SEO for quite sometime now, however one thing that always get me is URL structure. I have a decision to make, its either: URL 1 /conditions/allergies/food/ URL 2 /conditions/allergies-food/ Lets say i am optimizing for the key-phase "Food Allergies" what do you think is best practice? I know that this is not a major factor in gaining high SERPs & maybe i'm thinking about it too much, however your input would be really helpful. Kind Regards,
On-Page Optimization | | Paul780 -
Number of links in breadcrumb
Does google take the number of links in breadcrumb as a indicator to the depth of the page in the site structure? That will mean that a page with 5 links in the breadcrumb is a page very far away from the index, therefore a less important page.
On-Page Optimization | | seo.academy0