Why Did My Google Crawls Hit A Wall?
-
Hello,
One my the sites I work with, http://www.oransi.com, has seen a significant decrease in crawl Googlebot activity in the last 90 days. See screenshot.
This decrease in crawl stats runs in conjunction with less Kb downloaded per day & an increase in how much time it took Google to download a page.
The client did just go through a redesign, however that happened on 4/16/15, which was after the decrease in Googlebot activity, so that should not be the issue. Same could be said for the mobilegeddan algorithm change.
Any help would be greatly appreciated.
-
Hi BrandLabs,
Thanks for your question! You've had several community members respond, and they've asked some important questions. Did you see them? Please update, thanks!
Christy
-
Did they recently try and do some 301 redirects to point all non-www pages to the www version? It looks like the redirects may have been setup wrong and they're going to 404 pages instead of redirecting. There are a handful of other broken links on the site as well that could be contributing to the decline in Googlebot crawling. That's where I would start.
-
Well it's clear that something has happened.
Was any work carried out prior to to drop? Are you seeing any increase in 404's or problems in Google Search Console (Webmaster Tools after the daft re-brand)?
-Andy
-
Has there been any changes to website traffic or more specifically organic traffic over this same period?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Pagination Changes
What with Google recently coming out and saying they're basically ignoring paginated pages, I'm considering the link structure of our new, sooner to launch ecommerce site (moving from an old site to a new one with identical URL structure less a few 404s). Currently our new site shows 20 products per page but with this change by Google it means that any products on pages 2, 3 and so on will suffer because google treats it like an entirely separate page as opposed to an extension of the first. The way I see it I have one option: Show every product in each category on page 1. I have Lazy Load installed on our new website so it will only load the screen a user can see and as they scroll down it loads more products, but how will google interpret this? Will Google simply see all 50-300 products per category and give the site a bad page load score because it doesn't know the Lazy Load is in place? Or will it know and account for it? Is there anything I'm missing?
Intermediate & Advanced SEO | | moon-boots0 -
Reclaiming Ranking positions in Google
We have a website we are working on that was ranking well in Google but since having a hosting upgrade has completely dropped in rankings. When a hosting upgrade was made, the developer added an incorrect robots.txt file that restricted the site from being found, hence resulting in lost rankings. We have since sorted out that issue so the robots.txt is OK. However, ranking results have yet to be reclaimed. We are unsure why these rankings haven't rebounded back, as it has been a while now. The site is https://www.brightonpanelworks.com.au. We have since also attempted to add a sitemap however to help the site be better crawled and to regain rankings, however, it appears that sitemap generators are having problems creating a sitemap for this site and we are not sure why. And we are not sure whether this may relate to why Google has not picked up on pages and ranking results have not be restored. If you have any ideas as to how we can reclaim rankings to the strong positions they were in previously, that would be much appreciated. We believe we may be missing something here that is not allowing webpages to be picked up and ranked by Google.
Intermediate & Advanced SEO | | Gavo0 -
Google index
Hello, I removed my site from google index From GWT Temporarily remove URLs that you own from search results, Status Removed. site not ranking well in google from last 2 month, Now i have question that what will happen if i reinclude site url after 1 or 2 weeks. Is there any chance to rank well when google re index the site?
Intermediate & Advanced SEO | | Getmp3songspk0 -
Rotating content = Google Penalty?
Hi all. We have an ecommerce site which features various product sections. In each section you might have 60 products each displayed neatly in pages of 10. We recently added functionality, so that if a product is out of stock, it will automatically drop that product to the back of the list and bring another in stock one forward. We're just worried that Google will view the same information, repeatedly rotating on the first page of 10 products (the page that ranks) and think we're in some way trying to trick Google into thinking the content is fresh? Does anyone have a throw on this? Is it likely to penalise us? Thank you!!! Ben
Intermediate & Advanced SEO | | bnknowles10 -
Does putting a Google custom search box on make Google think my users are bouncing?
I added a Google custom search box to my pages, that's doing an advanced Google search. A lot of people are using it. So users are coming to my site from a Google search, and then often performing another Google search on my site. Should I be worried that Google may interpret the resultant user behavior as a bounce or pogo-stick? Or will the fact that the second search occurred on my site, using custom search, and with advanced parameters signal to Google that this is not a dissatisfied user returning to Google? Thanks
Intermediate & Advanced SEO | | GilReich0 -
Google Reviews
I have a couple of reviews from clients on Google that seem to have just disappeared. What gives?
Intermediate & Advanced SEO | | bronxpad0 -
Indexed Pages in Google, How do I find Out?
Is there a way to get a list of pages that google has indexed? Is there some software that can do this? I do not have access to webmaster tools, so hoping there is another way to do this. Would be great if I could also see if the indexed page is a 404 or other Thanks for your help, sorry if its basic question 😞
Intermediate & Advanced SEO | | JohnPeters0 -
Google WMT Change of address.
Hey all. I've got two domain names. -rt112media.com -route112media.com http://rt112media.com is my main address. The preferred domain. My Google WMT account has 4 sites listed that I am a verified owner on. 1.)rt112media.com 2.)www.rt112media.com 3.)route112media.com 4.)www.route112media.com I recent changed the address in WMT for route112media.com to have the preferred domain rt112media.com. I did this on Nov 23. It is currently saying their is a request and I can withdrawl. Now I'm needing to set the preferred for www.rt112media.com to be rt112media.com but it will not let me because of the pending route112media.com. The problem is I'm getting a ton of errors under my www.rt112media.com diagnostics. I'm not getting any under rt112media.com but I am thinking it is still effecting my page rank. I have the 301's for all set up as a 301 wildcard to rt112media.com. They all redirect to the preferred domain I wan (rt112media.com.) Suggestions?
Intermediate & Advanced SEO | | Route112Media0