Database Crash affecting our rankings
-
Here is one for you. We had a database crash, and was down for an hour and 15 minutes, on Dec 18th. During this time our system automatically sent out a 301 redirect to our product home page. (iboats.com/boat-parts-accessories/dm/)
On Dec 24th I noticed a huge ranking drop on a lot of our keywords, and some keywords that were serving up the redirect page from our database crash, in place of its normally ranked page.
So my assumption is that Google was crawling our site during the database crash and caching the redirect pages in place of our regularly ranked pages. So when a search query is made that would normally display our ranked page, it is now displaying the redirect page from the crawled cache.
And on other keywords where we normally would rank for, Google no longer has that page cached, so it drops it from ranking.
My question: Does this assumption sound accurate? If so, I'm assuming over time that our past ranked pages will show up once again after Google recrawls those pages and saves them in their cache.
I have several sites that were affected with this:
Thanks in advance for your input
-
Glad to hear you appear to be back on the right track
-
Update:
After unblocking Google, almost all of our rankings are back up, and in some cases a little improved. I was surprised how quickly Google recrawled and ranked the pages that dropped.
I did do a Fetch and submit on all my websites and pages. I'm not sure how Fetches are attended to by Google, but it appears they are.
I have an IT guy who doesn't feel a Fetch makes any difference. He says Google will crawl when they want to and probably does not pay attention to the Fetch request.
-
I wanted to get back to report the cause of our problem with a lot of our rankings dropping. We originally thought it was due to a database crash. And that still could be affecting some things.
We moved our servers to a new location. A couple of weeks ago, we switched over to some new IP addresses. When we did so, a program started blocking Google's IP addresses because they were hitting our servers with a lot of hits, and put those IP addresses on a blocked list. So our pages on some of the websites stopped getting crawled, and thus rankings dropped. We discovered the problem when exploring our Webmaster Tools Crawl Stats and noticed that the "kilobytes downloaded per day" had flat-lined. Doing a fetch on selected pages we could see that it was not fetching the page. So after unblocking those IP addresses, the pages got crawled, our crawl stats went back up, and our rankings started going back to where they were. We still have some rankings that have not bounced back, but assume they will as new crawls re-index those pages.
A lesson learned for me was to pay attention to the crawl stats. Any drastic variations could be a sign of a problem.
Brad
iboats.com -
I'm pretty sure that GWT is a little vague in terms of times that Googlebot visited with just the crawlrate shown on a day by day basis and there's some lag between now and when it happened. CPanel will show when spiders visited in latest visitors from memory, however I think that rather than looking back it would be better to simply ensure that this doesn't happen again with the 302 rather than the 301. Maybe refreshing of your sitemap submissions in GWT would help to get some crawling going on with a ping or two to get Googlebot back soon.
-
Also, can't we use GWT to find out the exact times or at least approximate times of when Google crawled a site? Or can we not find that information so precisely? In this case, it does seem that the 301 vs. 302 is the key problem, but also the fact that the system for getting rid of the 301 didn't work out properly.
-
It seems extreme that Google would react within an hour and 15 mins but who knows nowadays. Keep an eye on your GWT account and hopefully if you have a decent crawl rate things should remedy themselves. Stephen Salstrand made a good point. 302's rather than 301's should help. It's great to have something in place like a redirection for user experience in case of database failure, however, as Stephen said, it should be made clear to Google that it's only temporary.
-
Investigating it further, we have some permanent redirects that are still hanging. I've got my IT guys working on it as we speaking.
I've got to run, but I would like to revisit this soon. Once we have determined the problem, I will post it here.
Thanks for your opinions. They have helped.
-
Your assumptions are globally right. However, I have never seen Google react that quickly to a 301 redirect or even a crash, I always tell my customers "if you have a temporary issue with 404 and 500 erros, don't worry about it too much, Google won't react unless it is a permanent issue that lasts over three or four crawls".
This is why I would also investigate a penalty that just happened to take effect shortly after the crash. To rule that out, could you please tell us if some of your major keywords are still ranking well?
I don't think you sould worry too much about that, just wait a few days and see if some pages recover.
-
Good point on the 301's. I'll look into that.
Thanks, for your input!
-
I think your assumptions are on par. However, make sure to review that the 301 has been removed. Also, if you still want that auto redirect to happen on a crash, make it a 302 and not a 301 as a 301 will suck a lot of life out of your site (since SEs see it as permanent).
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Does having a sub-domain on a different server affect SEO?
I'm working with a company that has a hard-coded website on the root domain, and then a WordPress blog on a subdomain on a separate server. We're planning on implementing a hub and spoke model for their content, hosting the main hubs on the root domain and the linked articles on the blog. Is having the blog on a different server going to hinder our SEO efforts?
Technical SEO | | KaraParlin0 -
Knowledge Graph product carousel: The mystery of ranking
When you search for software-related terms, Google serves a carousel-style item above the organic listings. I've been assuming this is part of the Google Knowledge Graph. See http://imgur.com/a/TQpM5 The mystery lies in the selection of companies that are included in this carousel: the brands included are completely unrelated to the brands that rank well in the "normal" organic search listings. Giving that more color, here are the top 5 brands in "normal" organic search listings, for the query "invoice software," noting where they appear in the carousel: BRAND // normal organic search position // carousel position Wave // 1st // not present in carousel Quickbooks // 2nd // 1 NCHSoftware // 3rd // not present in carousel Invoicely // 4th // not present in carousel Freshbooks // 5th // not present in carousel Inversely, here are the top 5 in the carousel, noting where they appear in the "normal" organic search listings, again for the query "invoice software": BRAND // carousel position // normal organic search position Quickbooks // 1st // 2nd Zoho Office // 2nd // 9th Invoicera // 3rd // 15th Hiveage // 4th // 30th Apptivo // 5th // not present in first 10 pages of search results Here's an annotated image of the same thing: http://imgur.com/a/0Pa6j
Technical SEO | | RobM416
--I represent a brand (I'd rather not mention which) that ranks super well in the normal organic search listings, but doesn't rank accordingly in the knowledge graph carousel. WHAT WE'VE TRIED implemented Schema markup updated Wikidata information updated Wikipedia information We've reviewed the code on the pages that DO appear in the carousel, and haven't found any common threads. Several weeks have passed since those initiatives. We've been re-indexed in that time, but are still not included in the search carousel. I've seen people suggest resources like this
https://searchenginewatch.com/sew/how-to/2299454/4-google-carousel-optimization-tips
but those tips aren't addressing this problem. Also, things like claiming your business or other local business validation (which we've done) don't seem material to a non-local product search (but stranger things have happened, I know...). A couple of SEOs have suggested AMP, but I've yet to see any authoritative info suggesting a link between AMP and desktop results. Any help appreciated! I will report back on my findings. This is a follow-up to a question I previously posted here: https://moz.com/community/q/kinds-of-organic-search-results-google TQpM5 0Pa6j0 -
Changing images on site without losing ranking
A number of images on my site rank very well under google image search but need to be replaced with updated versions. If I keep the file name and pixel dimensions identical will switching the image effect my rankings? Thanks!
Technical SEO | | Justin450 -
Sudden decline in ranking
Hi SEO People, Rankings for one of my search keywords has declined over the last two weeks from place 2-4 down to 13-15 on Google NZ. It had sat at the place 2-4 level for months. Nothing has changed on any of the sites above/below ours, and nor have any changes been made to our site. All our other, similar, keyword rankings have either stayed unchanged or improved. Does anyone have any bright ideas?
Technical SEO | | chris.elevate0 -
Rankings of Subdomains vs. Main Domain
Here's a puzzler... Our main domain (www.ides.com) doesn't appear in Google (but does on Bing and other engines). We think this is due to duplicate content which we're fixing. However our website's subdomains continue to appear prominently in SERPs, even on Google - here are some examples: IDES Prospector = prospector.ides.com IDES = support.ides.com Cycolac FR15 = catalog.ides.com Why would Google penalize a main domain and its subdomains?
Technical SEO | | Prospector-Plastics0 -
Whats happened to my rankings
Hey everyone, I hope someone might be able to help me figure out what has gone wrong here. I have a website which was slowly growing in rankings. The domain used was a .net However, It is a UK based business so I shifted this to a .co.uk domain and then followed googles advice in webmaster tools for moving address. ie, copy site, verify in GWMT and then 301 the old site. Now the .net had 71 indexed pages out of 73. But after 2 months .co.uk only has 41 out of 73 indexed. At the end of last week I did change the theme on my website although the content / urls have all remained the same. I have gone to check my rankings this morning and the site has totally bombed. Even the own brand name does not ranking in top 10 pages. I have not done an major link building at all yet for the site and any links that are there are all high quality from industry related authority sites. Looking at GWMT the latest links all look ok, nothing spammy but one thing I did notice is that it shows the .net domain providing 193 links via 301and also individually a great deal of the links that point to .net but have shown in .co.uk What I would like to know if have I actually done something wrong? Should I just kill the 301 (i am loathed to do this as a few of the links to this took me a long time to get), after spending a great deal of money developing my site and just starting to see some nice progress in the natural rankings and google has gone and pulled the carpet from under me. I would understand if I was doing something wrong but I have made strenuous efforts to ensure the linking on this site is whiter that white. Oh, I have had no warnings in my messages at all. Thanks in advance for any advice.
Technical SEO | | easyrider20 -
Why am I ranking for this
In need of some second opinions here. I have a domain softboxsystems.co.uk which ranks for the keyword sofrigam. It is not meant to! Have posted previously on this but to cut long story short, the clients .com started to rank for this term and then they lost the contract to supply the services so got a legal request to stop ranking for it (it had been seoed so was ranking 100% on off page issues). The client is a UK company so they decided that they would block the .com from google and use the co.uk domain for a period of time instead while they worked on deranking the domain. The .com domain was therefore blocked from the index, the .co.uk homepage would not rank for anything then it turned out that they had placed a canonical tag link from the .co.uk to the blocked .com!!! On my suggestion this was removed and the homepage of the .co.uk appeared in google in a matter of a couple of days (other pages on the site were indexed but not the home). Now, somehow the co.uk domain homepage is ranking for the banned term. I have checked open link and there are not any links pointing to the domain with the anchor of sofigram, nor is the term mentioned onsite. I now have the client panicking on the phone to get the page out of google again asap before the competitors legal people get wind of it. I am still trying to deindex the .co.uk homepage in the short term. I have readded the canonical back to the .com as that worked at not getting that page indexed previously. The main issue I have is how on earth I am ranking for this term in the first place and more importantly how do I stop from ranking for this term! The only terms I can find as anchor text in the moz tools are the keywords I have been seoing for help hehe thanks
Technical SEO | | Grumpy_Carl0 -
Duplicate content domains ranking successfully
I have a project with 8 domains and each domain is showing the same content (including site structure) and still all sites do rank. When I search for a specific word-string in google it lists me all 8 domains. Do you have an explanation, why Google doesn't filter those URLs to just one URL instead of 8 with the same content?
Technical SEO | | kenbrother0