Why Did My Google Crawls Hit A Wall?
-
Hello,
One my the sites I work with, http://www.oransi.com, has seen a significant decrease in crawl Googlebot activity in the last 90 days. See screenshot.
This decrease in crawl stats runs in conjunction with less Kb downloaded per day & an increase in how much time it took Google to download a page.
The client did just go through a redesign, however that happened on 4/16/15, which was after the decrease in Googlebot activity, so that should not be the issue. Same could be said for the mobilegeddan algorithm change.
Any help would be greatly appreciated.
-
Hi BrandLabs,
Thanks for your question! You've had several community members respond, and they've asked some important questions. Did you see them? Please update, thanks!
Christy
-
Did they recently try and do some 301 redirects to point all non-www pages to the www version? It looks like the redirects may have been setup wrong and they're going to 404 pages instead of redirecting. There are a handful of other broken links on the site as well that could be contributing to the decline in Googlebot crawling. That's where I would start.
-
Well it's clear that something has happened.
Was any work carried out prior to to drop? Are you seeing any increase in 404's or problems in Google Search Console (Webmaster Tools after the daft re-brand)?
-Andy
-
Has there been any changes to website traffic or more specifically organic traffic over this same period?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Copyright Theft and Google Rankings
Here is another tough one we've been dealing with. We publish a niche book. For a decade we kept the information offline (no e-books). However, it was widely scanned and reproduced online. We've filed dozens of DMCA complaints over the years, but have found trying to rid the internet entirely of these infringing pages to be futile. We get 1 closed and find 3 more. Two years ago we decided to put the information online ourselves, to generate an official community for our work it instead of "fighting it". We built a full site with hundreds of pages from the book for readers to use, free. Google indexed us, and we followed basic SEO... But in spite of a prime aged domain and a lot of links, we are literally BURIED in google. There are dozens of complete garbage spam sites that rank way higher than us. I understand ranking takes time, and the niche is competitive. But the low quality landing pages that are ranking above us is just too confusing. We fear our work has been indexed by google so much over the years on other sites they will never connect it to us. We'll always be buried on page 14 as another scrape. What would you do to correct this for a client? Could you?
Intermediate & Advanced SEO | | RetBit0 -
"Null" appearing as top keyword in "Content Keywords" under Google index in Google Search Console
Hi, "Null" is appearing as top keyword in Google search console > Google Index > Content Keywords for our site http://goo.gl/cKaQ4K . We do not use "null" as keyword on site. We are not able to find why Google is treating "null" as a keyword for our site. Is anyone facing such issue. Thanks & Regards
Intermediate & Advanced SEO | | vivekrathore0 -
Google is not indexing an updated website
We just relaunched a website that has 5 years old, we maintain all the old URLs and articles but for some reason google is not picking up the new website https://www.navisyachts.com. In Google Webmaster Tools we can see the sitemap with over 1000 pages submitted but shows nothing as indexed. The site is loosing traffic rapidly and positions, from the SEO side all looks fine for me. What can be wrong? I’ll appreciate any help. The new website is built over Joomla 3.4, we have it here at MOZ and other than some minor details it doesn't show that something can be wrong with the website. Thank you.
Intermediate & Advanced SEO | | FWC_SEO0 -
How to find all indexed pages in Google?
Hi, We have an ecommerce site with around 4000 real pages. But our index count is at 47,000 pages in Google Webmaster Tools. How can I get a list of all pages indexed of our domain? trying to locate the duplicate content. Doing a "site:www.mydomain.com" only returns up to 676 results... Any ideas? Thanks, Ben
Intermediate & Advanced SEO | | bjs20100 -
Dropped Out of Google and Bing
I am helping with a site that at one time I had on page 1 for Google/Bing. Site started to slip in rankings, then someone else did a makeover of the store and botched things by renaming pages, having errors in pages (multiple head/body), mismatch page names from sitemap, etc. Site slipped to page 4/5. I righted things, fixed duplication using canonicalization, made some other changes. Now site is gone completely from Google/Bing for desired keyword. No penalties. Site still shows if do search on domain name. Site is www.plussizeplum.com (plus size lingerie, sorry), keyword target is plus size lingerie. Anyone have any clues, tips, etc on why we fell off the face of the earth? Page Authority/Domain Authority are both comparable to most of the page 1/2 sites for same thing. Thanks for any advice.
Intermediate & Advanced SEO | | dlcohen0 -
How to determine URL Parameters in Google Webmaster
Hi there! I have a new website with so many duplicate meta titles and descriptions because of its expanded features from the e-commerce shopping cart that I am using like mobile website, product sorting, etc. Aside from canonical, is it advisable to use the URL parameters from Google webmaster tools to disallow crawling of mobile website and other parameters like, "parent", "catalogsetview", "pcsid", "pg" "mode". I appreciate and advise. 🙂 Thanks!
Intermediate & Advanced SEO | | paumer800 -
How to Block Google Preview?
Hi, Our site is very good for Javascript-On users, however many pages are loaded via AJAX and are inaccessible with JS-off. I'm looking to make this content available with JS-off so Search Engines can access them, however we don't have the Dev time to make them 'pretty' for JS-off users. The idea is to make them accessible with JS-off, but when requested by a user with JS-on the user is forwarded to the 'pretty' AJAX version. The content (text, images, links, videos etc) is exactly the same but it's an enormous amount of effort to make the JS-off version 'pretty' and I can't justify the development time to do this. The problem is that Googlebot will index this page and show a preview of the ugly JS-off page in the preview on their results - which isn't good for the brand. Is there a way or meta code that can be used to stop the preview but still have it cached? My current options are to use the meta noarchive or "Cache-Control" content="no-cache" to ask Google to stop caching the page completely, but wanted to know if there was a better way of doing this? Any ideas guys and girls? Thanks FashionLux
Intermediate & Advanced SEO | | FashionLux0 -
Google indexing flash content
Hi Would googles indexing of flash content count towards page content? for example I have over 7000 flash files, with 1 unique flash file per page followed by a short 2 paragraph snippet, would google count the flash as content towards the overall page? Because at the moment I've x-tagged the roberts with noindex, nofollow and no archive to prevent them from appearing in the search engines. I'm just wondering if the google bot visits and accesses the flash file it'll get the x-tag noindex, nofollow and then stop processing. I think this may be why the panda update also had an effect. thanks
Intermediate & Advanced SEO | | Flapjack0