Quickest way to deindex a large number of pages
-
Our site was recently hacked by spammers posting fake content and bringing down our servers, etc. After a few months, we finally figured out what was going on and fixed the issue. However, it turns out that Google has indexed 26K+ spammy pages and we've lost page rank and search engine rankings as a result.
What is the best and fastest way to get these pages out of Google's index?
-
Given that I'm sure you've removed these pages from your site, there will be no page to which to add a meta-noindex tag.
Disallowing these pages in robots.txt in no way signals to the search engines that they should be removed from the index, just that they should no longer be crawled. Given that they're already indexed, blocking in robots.txt would potentially save some "crawl budget" but wouldn't do anything to remove them from the index.
So submitting them to the URL Removal Tool would be by far the most effective, along with an explanation.
You'll also want to keep a very close watch on your penalty warnings within Webmaster Tools. If you get flagged, you'll want a complete history of the issue and the steps you've taken to address it in order to prepare a reinclusion request.
Lastly, don't forget to submit these same URLs to the Bing Webmaster Tools Block URLs tool. You may not get a massive amount of traffic from Bing, but there's no sense throwing it away, since you've already prepared the URL removal list anyway.
Hope that helps?
Paul
-
Yup. Just wanted to add as well that if these pages are in a particular directory, then you can deindex the entire directory in one command using the URL removal tool.
-
Disallow in robots.txt
Add a noindex meta tag to these pages
Request Google to remove the URLs from their index via WMT URL removal request
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Number or percentage of new visitors impact Google rankings?
Hi all, Does the number/percentage of new visitors (from different IPs and countries) impact the Google rankings? If there are more number of new visitors, will Google favours the website in rankings considering the fact that new visitors are better than returning/same visitors? Thanks
Algorithm Updates | | vtmoz0 -
What happens when we canonical and point to a page which has been redirected to another page? Google response!
Hi all, I would like to know the different scenarios Google going to respond when we use canonical and redirect for duplicate pages. Let's say A to B are duplicate pages with 95% same content and C Doesn't have same content but context wise similar and priority page we expect to rank for. What happens if we canonical from A to B and set redirect from B to C? What if both A and B are pointed to C with canonical? What if A or B deleted and other one is canonical to C? Note: We can noindex or 301 redirect as they have their own visitors. This is more about showing most relevant content to the audience and avoid duplicate content in search results. Thanks
Algorithm Updates | | vtmoz0 -
Google AMP (accelerated mobile pages), can it be used for non-Google news and Ecommerce Websites?
Mozzers, I've been doing a lot of research on Google's new Accelerated Mobile Pages (AMP) https://moz.com/blog/accelerated-mobile-pages-whiteboard-friday. From what I'm seeing, these AMP version websites are only for Google News-worthy websites such as New York Times, Cosmopolitan, and the BuzzFeeds of the world. But what about Ecommerce websites like Ebay or Amazon? Will AMP versions of "scotch tape" via OfficeDepot work in the SERP's on non-Google News cards?
Algorithm Updates | | Shawn1240 -
Website dropping from Page 1 pos 5 to no ranking and then back again?
Hi all, We have a very odd occurrence with a client of ours. It should be noted that they had a penalty recently removed about 2 months ago after much work from our company. Recently they started appearing back on Page 1 Google for a semi competitive keyword term. We were very happy with this and so was the client. The the ranking improved with our work to position 5, which was excellent. Unfortunately what has been happening is they have been dropping out of the rankings completely for this semi competitive keyword for a few days and then reappearing in the same position. The client is checking daily and has noticed. I thought this is just a 'hangover' from the Google penalty and perhaps a one off occurrence, but it has happened about 3 or 4 times now and seems to be happening every couple of weeks. Can anyone shed some light on this behavior? I have checked Webmasters Tools and everything is fine. Thanks Jon
Algorithm Updates | | Jon_bangonline0 -
How long can your header be in number of characters?
hey all, have hear conflicting information on how many characters in the header you are allowed? i've heard 60, 65, and 70?? what are your thoughts? thanks!!
Algorithm Updates | | tm46150 -
Has anyone notice their website page rankings in Bing and Yahoo slipping lately?
My websites for the most part rank higher in Yahoo and Bing than in Google. Now, ranking within those search engines (Yahoo/Bing) are settling in to where my websites rank in Google. Any thoughts? Anyone else experiencing this? Thank you for your time!
Algorithm Updates | | APICDA0 -
Best way to find new studies?
I'm wondering how sites break news without dedicating too much time researching. Are there any tools that you use? For example, if you wanted to be one of the first people to know when a celebrity was diagnosed with a health condition, how would you go about doing that?
Algorithm Updates | | nicole.healthline0 -
What's the best way to discover which search terms competitors are highly-ranked for?
I'd like to know for which search terms competitors appear in the top 10, but I haven't found an efficient way to do so. Any help is appreciated...thanks!
Algorithm Updates | | actionagainsthunger0