Googlebot found an extremely high number of URLs on your site:
-
Website: www.gobol.in
Although I have no indexed my search pages by adding /catalogsearch in robots.txt, still we are getting same error again and again
Here's a list of sample URLs with potential problems.
- http://www.gobol.in/catalogsearch/result/index/?category=&mobile_feature=4575_4578&q=panasonic+NR-BU303LH1H+REFRIGERATOR+296+L+GREY&special_price=32%2C456&x=0&y=0
- http://www.gobol.in/mobile-and-accessories/mobiles-and-brands.html?manufacturer=4753_3355_455_4435_4720_3407_2412_4728_4784_4790_2010_4789_4376_2469&operating_system_mobile=4612
Please help
-
Thanks Dirk
-
Hi Ravi,
Did you use the robots.txt tester in Webmastertools - your robots.txt mentions to block /catalogsearch - if you want to block the entire directory I think you have to use
Disallow: /catalogsearch/
On top of that /catalogsearch is only blocked for Googlebot-image - not for the "normal" Googlebot - if you want to have it blocked for all bots you have to move it to the beginning of the file. Idem for /admin => should be /admin/ and in the beginning of your robots.txt file
rgds,
Dirk
-
Thanks Martijn
-
Hi Ravi,
This message you got from Google is more a warning then an error, that is good to keep in mind. As there is no problem in theory with having a lot of URLs on your site if the quality is high enough. Therefore the first URL shouldn't cause any troubles, as like you mention, it's included in the robots.txt file.
The second URL is a bit less nice but shouldn't also be trouble, as it's making sure that the URL parameters are working. Also there is a canonical URL on all these filter pages to make sure the pages are originally linked back to the category page. It could be worth it to check the URL parameters within Google Webmaster Tools to make sure Google is aware of all the changes in content for different parameters.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Home page keyword in url
I have been looking into SEO for a few weeks now trying to perfect a homepage. Going through various sources on MOZ, and other examples out there on the internet, I keep seeing that you should have your keyword in the URL of the page. The homepage is the page most people want to rank the highest in google searches, however, you cannot put the keyword in the URL as most home page URLs are simply /. Should I actually make the home like this: www.example.com/key-word-example? I would imagine this would not be the normal for many users and would seem like it's not the home page.
On-Page Optimization | | Matthew_smart0 -
Duplicate content on events site
I have an event website and for every day the event occurs the event has a page. For example: The Oktoberfest in Germany the event takes 16 days. My site would have 16 (almost)identical pages about the Oktoberfest(same text, adres, photos, contact info). The only difference between the pages is the date mentioned on the page. I use rich snippets. How does google treat my pages and what is the best practice.
On-Page Optimization | | dragonflo0 -
Recommendation: Add a canonical URL tag referencing this URL to the header of the page.
Please clarify: In the page optimization tool, seomoz recommends using the canonical url tag on the unique page itself. Is it the same canonical url tag used when want juice to go to the original page? Although the canonical URL tag is generally thought of as a way to solve duplicate content problems, it can be extremely wise to use it on every (unique) page of a site to help prevent any query strings, session IDs, scraped versions, licensing deals or future developments to potentially create a secondary version and pull link juice or other metrics away from the original. We believe the canonical URL tag is a best practice to help prevent future problems, even if nothing is specifically duplicate/problematic today. Please give example.
On-Page Optimization | | AllIsWell0 -
Altering site structure
I work for a business that operates several sites that were developed a very long time ago. We've been making many different changes over the past 12-18 months to improve these sites in several different ways. One area that we've never discussed or attempted is general site structure. Its pretty obvious that when the business was started they had never heard of information architecture or usability design. To make matters worse, the internal linking strategy appears to have been link everything to everything. Well after being told that it couldn't be done - I'm getting our team to say we must focus on this, if for no other reason that to help consumers figure out how to navigate through our site. Today we essentially have a series of category / information pages. In some cases, we hang more detailed topical content related to a category /informational page in a hub and spoke manner. Although remember what I said about linking everything to everything. In reality there are a series of subtopics that should been designed for every category / informational area. Instead, what happened is in some cases the subtopic is integrated into the hub or category page, in other situations is hung off the page as a spoke page and in others the subtopic isn't even covered. The plan is to standardize - each category will have 'n' subtopics (~10-12, we're still working this out). From a navigational standpoint users will be able to easily navigate both across categories as well as subtopics within a category as well as between categories within adjacent/similar subtopics. This is essentially a grid if that makes sense. The question is this - we have some keywords that do well in SEO and many many more that do not and the trend has not been our friend. We're considering keeping the URLs of the pages associated with strong keywords the same within the nav structure, even though this might mean the URL for a spoke page will be inconsistent with the spoke page name from a different category. I don't see any real danger for pages that either are not associated with any ranking keywords or only very weak keywords. Maybe I'm wrong. What things should we consider in this change? We believe that this standardization should help consumers find the information they are looking for in a much more efficient manner, so page views/visit should go up. Additionally, this prepares us for category and subtopic comparison pages and other added functionality being added in a logical manner. We also think that as we add depth about a subtopic, it will be easier for us to acquire links to our site because the subtopics within a category will appeal to different websites. This is by no means a small project. We have hundreds and hundreds of pages. Do folks think this is a worthwhile endeavor? We've spent a lot of time cleaning up H1 tags, structure of our pages, anchor tags, page load order and speed, image caching, etc. Site structure, URL length and internal link structure are essentially what is left. Once these are done we intend to really get going on better and more organized content on our site. Thoughts?
On-Page Optimization | | Allstar1 -
SEF URLs. Should I use / or - ?
I have o activate SEF URLs in a website. Regarding SEO, is there any difference between using / or - ? I mean, Is it better to write URLs like this: http://www.domain.com/folder/folder/page or like this: http://www.domain.com/folder-folder-page ? Is there any difference? Thanks!
On-Page Optimization | | ociosu0 -
Brand Name URL Redirecting to Actual URL
So we have already built a site under a parent company's URL: parentcompany.com And now we have their branded product lines in directories: parentcompany.com/brand-name1, and parentcompany.com/brand-name2 We also own the actual URL Brand Name 1 (which is also the exact description of the product): brandname1.com We do not yet own the URL for Brand Name 2 (which is also the exact description of the product): brandname2.com. This is because a squatter is sitting on it and is asking $10,000+ for it. What we are trying to determine is how valuable these brand name URLs are since they will be redirecting and not the actual site's primary domain name. Anybody know how much of an effect owning those and redirecting has on ranking for those brand names that are also very descriptive of the products? Would we be smarter to spend $10,000 on adwords or 10,000 on the domain? Thanks!
On-Page Optimization | | grayloon1 -
Monitor site loading speeds
We all know that site loading speed is important for google ( and getting more important), and currently we are considering moving to a dedicated server. But I would like to measure pur current loading times and compare them to our competitors. I know that webmaster tools shows you your loading times, but its seems a bit random with its results, and I know analytics also now records speed, but the sample sizes seems to be too low. When I googled website load times, I got a load of different free tools but they seem to give conflicting results e.g one said that our avg kb load time was 0.02sec and our competitor was 0.01, but a different site says our sites loads in 1.7 secs and our competitor loads in 11sec (our home pages have similar total file size for our homepage. ) Also what would be useful would be a way to monitor of website loading speed to see when it slows down, so we can figure out why. Does anyone know of any good reliable programs or sites that does this?
On-Page Optimization | | eunaneunan0 -
Is having the word catalog in an ecommerce site url detrimental to seo.
IS: www.example.com/catalog/category%/product% better than www.example.com/category%/product% category and product are dynamic values that change with the diff. categ. and products displayed while catalog is constant.
On-Page Optimization | | no6thgear0