Quickest way to deindex a large number of pages
-
Our site was recently hacked by spammers posting fake content and bringing down our servers, etc. After a few months, we finally figured out what was going on and fixed the issue. However, it turns out that Google has indexed 26K+ spammy pages and we've lost page rank and search engine rankings as a result.
What is the best and fastest way to get these pages out of Google's index?
-
Given that I'm sure you've removed these pages from your site, there will be no page to which to add a meta-noindex tag.
Disallowing these pages in robots.txt in no way signals to the search engines that they should be removed from the index, just that they should no longer be crawled. Given that they're already indexed, blocking in robots.txt would potentially save some "crawl budget" but wouldn't do anything to remove them from the index.
So submitting them to the URL Removal Tool would be by far the most effective, along with an explanation.
You'll also want to keep a very close watch on your penalty warnings within Webmaster Tools. If you get flagged, you'll want a complete history of the issue and the steps you've taken to address it in order to prepare a reinclusion request.
Lastly, don't forget to submit these same URLs to the Bing Webmaster Tools Block URLs tool. You may not get a massive amount of traffic from Bing, but there's no sense throwing it away, since you've already prepared the URL removal list anyway.
Hope that helps?
Paul
-
Yup. Just wanted to add as well that if these pages are in a particular directory, then you can deindex the entire directory in one command using the URL removal tool.
-
Disallow in robots.txt
Add a noindex meta tag to these pages
Request Google to remove the URLs from their index via WMT URL removal request
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Key webpage fluctuating between page 2 and page 6 of Google SERP
Hi, We have found that one of our key webpages has been fluctuating between page 2 and page 6 of Google SERP for around 2 weeks. Some days it will be on page 6 in the morning and then page 2 in the afternoon. We have recently updated some copy on the page and wondered if this could be the cause. Has anyone else experienced this? If so how long was it before the page settled? https://www.mrisoftware.com/uk/products/property-management-software/ Thanks.
Algorithm Updates | | nfrank0 -
Increase of non-relevant back-links drop page ranking?
Hi community, Let's say there is a page with 50 back-links where 40 are non-relevant back-links and only 10 are relevant in-terms of content around the link, etc....Will these non-relevant back-links impact the ranking of the page by diluting the back-link profile? Thanks
Algorithm Updates | | vtmoz0 -
Does Google giving more important to internal pages than homepage recently? Especially after the recent Major algo update?
Hi everybody, I can see the change Google brought in the SERP. Previously website homepages will be shown for primary keywords, now it's slowly and almost switched to showing most related internal pages in a website. You can check same for keyword "SEO", Most or all the results are internal pages. I can see this change for our primary keyword from last one month. So basically Google is trying to show a page explaining about the primary keywords rather than website, that's how "what is seo" pages are ranking than homepages. If there is no such pages existed or not well written, Google is just showing the website homepage. But I noticed that websites ranking with homepages are dropped compared to the websites with dedicated page about that primary keyword. Please share your thoughts. Thanks
Algorithm Updates | | vtmoz0 -
Backlinks to internal pages help website to rank better or vice versa?
Hi Moz community, We have our backlinks mostly pointed to our homepage. We are trying to rank better and not having minimum number of backlinks to our internal pages is one of the things I worry about. Backlinks to homepage alone help in ranking internal pages or backlinks to internal pages help in ranking homepage? Or both required? Thanks
Algorithm Updates | | vtmoz0 -
Header Structure In Product Gallery Page
Hi Everyone, Should product names have an H2 header tag on a gallery page? (H1 already optimized) Why or why not?
Algorithm Updates | | JMSCC0 -
Google Panda - large domain benefits
Hi, A bit of a general question, but has anyone noticed a improvement in rankings for large domains - ie well known, large sites such as Tesco, Amazon? From what I've seen, the latest Panda update seems to favour the larger sites, as opposed to smaller, niche sites. Just wondered if anyone else has noticed this too?Thanks
Algorithm Updates | | Digirank0 -
Has anyone seen this before? One domain dominates the entire first page!
Do a google search for "sober college" and tell me you don't see the entire page filled with one domain. (except the last result)
Algorithm Updates | | EmarketedTeam0 -
Not sure whether to update existing page or create a new one
Hello guys, So far I have found this Q&A to be very helpful. I have a couple of product pages on my website which rank very low in the search results. This was because in the past they were not at all optimized in terms of title, description, etc. I would like to now optimize these pages, get some inbound links to them, etc. I'm not sure whether to do this with the existing product pages or create new ones. I'm afraid that if I optimize the current pages, the low ranking from before will carry over. Would it be better to start fresh? Thank you, Pravin
Algorithm Updates | | goforgreen0