Google Dropped 3,000+ Pages due to 301 Moved !! Freaking Out !!
-
We may be the only people stupid enough to accidentally prevent the google bot from indexing our site.
In our htaccess file someone recently wrote the following statement
RewriteEngine On
RewriteCond %{HTTP_HOST} ^mysite.com$ [NC]
RewriteRule ^(.*)$ http://www.mysite.com/$1 [L,R=301]Its almost funny because it was a rewrite that rewrites back to itself... We found in webmaster tools that the site was not able to be indexed by the google bot due to not detecting the robots.txt file. We didn't have one before as we didn't really have much that needed to be excluded. However we have added one now for kicks really. The robots.txt file though was never the problem with regard to the bot accessing the site.
Rather it was the rewrite statement above that was blocking it. We tested the site not knowing what the deal was so we went under webmaster tools then health and then selected "Fetch as Google" to have the website. This was our way of manually requesting the site be re-indexed so we could see what was happening. After doing so we clicked on status and it provided the following:
HTTP/1.1 301 Moved Permanently
Content-Length: 250
Content-Type: text/html
Location: http://www.mystie.com/
Server: Microsoft-IIS/7.5
MicrosoftOfficeWebServer: 5.0_Pub
MS-Author-Via: MS-FP/4.0
X-Powered-By: ASP.NET
Date: Wed, 22 Aug 2012 02:27:49 GMT
Connection: close<title>301 Moved Permanently</title>
Moved Permanently
The document has moved here.
We changed the screwed up rewrite mistake in the htaccess file that found its way in there but now our issue is that all of our pages have been severely penalized with regard to where they are now ranking compared to just before the indecent.
We are essentially freaking out because we don't know the real time consequences of this and if or how long it will take for the certain pages to regain their prior ranks. Typical pages when down anywhere between 9-40 positions on high volume search terms. So to say the least our company is already discussing the possibilities of fairly large layoffs based on what we anticipate with regard to the drop in traffic. This sucks because this is peoples lives but then again a business must make money and if you sell less you have to cut the overhead and the easiest one is payroll.
I'm on a team with three other people that I work with to keep the SEO side up to snuff as much as we can and we sell high ticket items so the potential effects if Google doesn't restore matters could be significant. My question is what would you guys do? Is there any way we can contact Google about such a matter? If you can I've never seen such a thing. I'm sure the pages that are missing from the index now might make their way back in but what will there rank look like next time and with that type of rewrite has it permanently effected every page site wide, including those that are still in the index but severely effected by the index.
Would love to see things bounce back quick but I don't know what to expect and neither do my counterparts. Thanks for any speculation, suggestions or insights of any kind!!!
-
Interested to know if the indexing and rankings of the pages came back, David?
Paul
-
If every single page was set to 301 back to your home page and subsequently Google has removed all other URLs from the index, then disable the 301's, make sure the pages are working, and resend the sitemap to Google.
Do a scan with link sleuth to confirm all your pages are working correctly.
Greg
-
Thanks for the reply. Actually we were not trying to redirect the site anywhere. That was a redirect that literally redirected mysite.com to the identical mysite.com. Therefore we had no want or need to redirect anything other than internal url's. I think someone may have intended this to be used as a temporary overall site redirect during a maintenance schedule or something so that every page went to the same page but I assuming this was messed up because someone didn't put in the correct redirect code or url for that matter. Hope that makes sense.
So the issue was the google seen thousands of pages that were looking like a 301 redirect to the homepage and even the homepage redirected to the homepage as a permanently moved.
-
Ouch, sorry to hear that. I just added the htaccess code to one of my dev sites and it seems to redirect just fine (200 when visiting www, 301 when visiting non-www). I'm guessing that Google is going to the non-www page and is being redirected to the www version. Maybe switch the code to do the opposite (assuming your goal was to canonical/consolidate the urls). Message me if you want the htaccess code.
For fixing your site rankings, they will most likely bounce back on their own in due time so the best thing to do is try to get G to recrawl all the pages ASAP. I would:
-
Make sure you add and verify both the www and non-www urls of the site in GWT, then choose which one you want to be primary.
-
Change you sitemap to have have frequency/priority crawl rates, have the correct www/non-ww links and resubmit your sitemap in GWT.
-
Share links on twitter (they are crawled very often)
-
Create an rss feed with your pages and ping it
Good luck getting it ranking again.
-Oleg
-
-
First off a 301 redirect takes a few weeks (2-12) to fully work.
Second off you can send google a letter of reconsideration if you feel you were penalized. However I would give the 301 redirect a little boit of time to transfer your link juice. Also when you do a 301 i've heard that only 80% of your link juice will transfer. I like to contact my best links and have them link directly to the new site.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google creating it own content
I am based in Australia but a US founded search on 'sciatica' shows an awesome answer on the RHS of the SERP https://www.google.com/search?q=sciatica&oq=sciatica&aqs=chrome.0.69i59.3631j0j7&sourceid=chrome&ie=UTF-8 The download on sciatica is a pdf created by google. Firstly is this common in the US? secondly any inputs on where this is heading for rollout would be appreciated. Is google now creating its own content to publish?
Algorithm Updates | | ClaytonJ0 -
Google search analytics position - how is it worked out
In our Google search analytic s graphs total clicks and impressions appear as a sold line on the graph(ie showing a result for each day) Position only shows as an occasional dot or line - not a continuous result for each day) sometimes there are days with no result for position. How do google get these results
Algorithm Updates | | CostumeD0 -
Missing Keywords in Google SERP
We just got this attached image from one of our partners - has anyone seen Google putting 'missing' keywords in SERPs like this before? They said that it was not a plugin or anything and this is a screenshot of their organic search results. google%20screenshot_zpsgmwaf9e2.png
Algorithm Updates | | ReunionMarketing0 -
How long does google take to re-ranking pages in results?
I mean when google dance, the pages in results go up and down frequency every minue, but finally your page will rank in any position in google, what is the time when you get another position in google
Algorithm Updates | | engtamous0 -
301 a purchased domain
I've purchased a competitor. They rank well organically for keywords that I target, and I want to optimize the way I get value from their current rankings and traffic (and customers -- we will obviously market to their email/customer list). Which is better: (1) use a 301 redirect for any access to their domain and point it to my home page. I think this would force Google to de-index all of their pages, right? (2) put up a stub page as their homepage that announces the site has been bought, and have a do-follow link to my home page (which maybe is auto-redirected after 10 seconds or something)? Maybe this is better to keep their home page in Google's index for a while? As for option (1), I thought I read somewhere recently that 301'ing a domain to the home page of another domain would no longer pass link juice (?). Maybe I should 301 the newly purchased domain to a sub-page on my site that explains the acquisition and asks them to sign up on my site? Both sites are legit. No spamming happening here; just industry consolidation as one competitor acquires another. Thanks in advance...!
Algorithm Updates | | scanlin0 -
Dropped from Universal Result: Local
For quite some time our Google Places listing has been in the Universal Results...(for this keyword there is a 7-pack result). Which was great, we had a PPC ad at the top of the page, we were 3rd in the Universal Results (there was 3 places listings before the natural results)...and we were 6th in the natural results - meaning we were on the first page 3 times...which means a happy boss....and lots of traffic. The old places listing was linked to our new Google+ Page pending the eventual demise of places and the merge. The merge has happened, all information from the places listing has migrated (apart from reviews and photos??) and the places listing has been deleted (URL returns 404 error). Problem is now my Google + Page is not even within the first 2 or 3 pages of places results never mind in the Universal results. So it would appear the rank / authority that the places listing had...hasn't been transferred to the Google+ page. My competitors...who were in 1 + 2 in the universal results above the natural results and who have Google+ Pages with NOTHING on...bar their name, are still there! Why would I be dropped when my Google+ Page, has more info, more followers, more photos, more relevant content (they don't have any content ) than my 2 competitors. It seems I've been penalised....somebody suggested that I had the keyword twice in my "About" and twice in my "Introduction" info and that could be it. I thought the loss of the review might be it too...but neither of the businesses now occupying the first 3 spots..have any reviews at all. Anybody else suffered from this? Anybody any other suggestions to why I might have been dropped so dramatically in the places listings? (My SERP listing is unaffected for this keyword) Keyword being mentioned twice hardly seems like "stuffing"! I'm actually not too concerned about the places ranking....not a great driver of traffic...but appearing in the Universal Results did obviously drive traffic...and to appear in the Universal Results...I've now got about 30 positions to climb...... The whole Google+ Local / Google Places thing has been a nightmare from start to finish.... Thanks in advance for any help or advice!
Algorithm Updates | | MarbellaSurferDude0 -
Does google have the worst site usability?
Google tells us to make our sites better for our readers, which we are doing, but do you think google has horrible site usabilty? For example, in webmaster tools, I'm always being confused by their changes and the way they just drop things. In the HTML suggestions area, they don't tell you when the data was last updated, so the only way to tell is to download the files and check. In the URL removals, they used to show you the URLs they had removed. Now that is gone and the only way you can check is to try adding one. We don't have any URL parameters, so any parameters are as a result of some other site tacking on stuff at the end of our URL and there is no way to tell them that we don't have any parameters, so ignore them all. Also, they add new parameters they find on the end of the list, so the only way to check is to click through to the end of the list.
Algorithm Updates | | loopyal0 -
Server Down for Few Hours went from Page 1 to Page 6?
We were on Page 1 - our server went down for about 4-6 hours and then we dropped to page 6. Would the server being down for this amount of time affect our position? Any advice would be much appreciated.
Algorithm Updates | | webdesigncwd0