Crawl rate drop
-
Hi Guys,
I have a crawl rate drop in webmastertools and can't figure out way. In the last month I removed a lot o duplicate pages that we don't need anymore, there were at least 1.5 million pages. Can this be a motive?
-
Hi there.
Of course it can! The crawl rate is in pages per day, so if you remove pages (especially 1.5 million), there won't be as much to crawl
Also it can happen, if you have the same static pages and the crawl bot has crawled them all. Google doesn't crawl all pages all the time, they have limited resources. So, if you have launched or updated website recently and now not really updating it, you can see the change in crawl rate.
However, you can change crawl rate for 90 days, if you need them to crawl your website constantly (usually good for websites in process of reconstruction). Here: https://support.google.com/webmasters/answer/48620?hl=en
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Multi Store SEO Drop
I have two stores (thespacecollective.com and thespacecollective.com/us) and over the past month the keyword rank for the US store had dropped by half, while the UK store is relatively the same. The content is mostly the same, except the US site uses US spelling. I assumed that this would not be flagged as duplicate content because it is the same site, just serving two locations. I'd be interested to hear some thoughts on the reason for this drop and how I might fix it.
Intermediate & Advanced SEO | | moon-boots0 -
Crawl Depth improvements
Hi I'm checking the crawl depth report in SEM rush, and looking at pages which are 4+ clicks away. I have a lot of product pages which fall into this category. Does anyone know the impact of this? Will they never be found by Google? If there is anything in there I want to rank, I'm guessing the course of action is to move the page so it takes less clicks to get there? How important is the crawl budget and depth for SEO? I'm just starting to look into this subject Thank you
Intermediate & Advanced SEO | | BeckyKey0 -
Recent ranking drop followed by strange behavior in SERPS
Recently on the evening of August 5th almost all of the keywords our pages ranked highly for dropped by anywhere from 5 to 8 pages. The only activity during this time was an article that had been picked up by a major news outlet and then was apparently copied onto other sources with links to our domain and article. More puzzling though is rather than simply having the same page show up lower for a keyword, in a number of instances a different page is now shown for the result, often with less or no relevance to the keyword. In some cases, for a single keyword phrase we've seen as many as 10 different pages rotated throughout the day when performing a search. Prior to our rankings falling, we've never seen this behavior.
Intermediate & Advanced SEO | | BrianQuinn0 -
My site dropped 10 spot on Google last week... why ?
Hello I have launched my site in April last year. http://www.hobartphotographertasmania.com.au/ It took a while but I finally got to rank 2 on Google for "Hobart photographer" and rank 1 for "photographer Hobart Tasmania". Suddenly last week my site dropped to page 2 and I don't understand why. Was there a Google update ? If not, what can be the reason for such a sudden drop ? Thanks Loic
Intermediate & Advanced SEO | | loiclg0 -
MOZ crawl report says category pages blocked by meta robots but theyr'e not?
I've just run a SEOMOZ crawl report and it tells me that the category pages on my site such as http://www.top-10-dating-reviews.com/category/online-dating/ are blocked by meta robots and have the meta robots tag noindex,follow. This was the case a couple of days ago as I run wordpress and am using the SEO Category updater plugin. By default it appears it makes categories noindex, follow. Therefore I edited the plugin so that the default was index, follow as I want google to index the category pages so that I can build links to them. When I open the page in a browser and view source the tags show as index, follow which adds up. Why then is the SEOMOZ report telling me they are still noindex,follow? Presumably the crawl is in real time and should pick up the new follow tag or is it perhaps because its using data from an old crawl? As yet these pages aren't indexed by google. Any help is much appreciated! Thanks Sam.
Intermediate & Advanced SEO | | SamCUK0 -
Rankings dropped - Why?
At the end of November a client site dropped significantly in the rankings. The drop effected almost all the keyphrases we monitor. Historically the homepage has always ranked higher than the sub-pages - however now it seems Google is no longer ranking the home page, and instead ranking the sub-pages, just far far lower down. Any ideas what could cause this?
Intermediate & Advanced SEO | | cottamg0 -
My homepage dropped off of Google today? Any ideas?
My site, www.leaffilter.com, previously ranked #1 for searches of "leaffilter" and "leaf filter". Today the #1 ranking for "leaffilter" with Google is gone. Well, it's not gone on all internet connection. At home the #1 ranking is in place. At the office computer it is not anywhere near page 1. I'm testing searches logged in, not logged in, cache and cookies cleared, different browsers. Other, secondary pages, are appearing on page 1. It looks as if the homepage is just no longer indexed...but not on all internet connection. Any ideas what is happening? Does anyone have any troubleshooting tips?
Intermediate & Advanced SEO | | LF_digital0
