Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
How do you de-index and prevent indexation of a whole domain?
-
I have parts of an online portal displaying in SERPs which it definitely shouldn't be. It's due to thoughtless developers but I need to have the whole portal's domain de-indexed and prevented from future indexing.
I'm not too tech savvy but how is this achieved? No index? Robots?
thanks
-
Fastest way to do it would be through a combination of Google Webmaster Tools and Robots.txt / noindex,nofollow meta tags.
Firstly if your domain hasn't been verified within GWT, go through that process. Then proceed with the remove URL request within GWT. (Image Attached)
You will need to upload a robots.txt and/or implement the noindex meta tag within the header of the pages you no longer want indexed.
Example Robots.txt:
User-agent: *
Disallow: /offendingportal/Example Robots Meta tag:
That should be enough to get rid of the offending content from within the SERPS and prevent it from being re-indexed.
-
Hi TCE,
There are two options in my opinion for this which are the best (as their are several other options including META robots etc):
- Robots.txt: Exclude your portal via your robots.txt file.
- HTTP Auth: Let your developers add HTTP Authentication so you have to log in to see the portal. Google won't be able to see the pages itself and so will de index them.
Hope this helps.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Does redirecting from a "bad" domain "infect" the new domain?
Hi all, So a complicated question that requires a little background. I bought unseenjapan.com to serve as a legitimate news site about a year ago. Social media and content growth has been good. Unfortunately, one thing I didn't realize when I bought this domain was that it used to be a porn site. I've managed to muck out some of the damage already - primarily, I got major vendors like Macafee and OpenDNS to remove the "porn" categorization, which has unblocked the site at most schools & locations w/ public wifi. The sticky bit, however, is Google. Google has the domain filtered under SafeSearch, which means we're losing - and will continue to lose - a ton of organic traffic. I'm trying to figure out how to deal with this, and appeal the decision. Unfortunately, Google's Reconsideration Request form currently doesn't work unless your site has an existing manual action against it (mine does not). I've also heard such requests, even if I did figure out how to make them, often just get ignored for months on end. Now, I have a back up plan. I've registered unseen-japan.com, and I could just move my domain over to the new domain if I can't get this issue resolved. It would allow me to be on a domain with a clean history while not having to change my brand. But if I do that, and I set up 301 redirects from the former domain, will it simply cause the new domain to be perceived as an "adult" domain by Google? I.e., will the former URL's bad reputation carry over to the new one? I haven't made a decision one way or the other yet, so any insights are appreciated.
Intermediate & Advanced SEO | | gaiaslastlaugh0 -
Why some domains and sub-domains have same DA, but some others don't?
Hi I noticed for some blog providers in my country, which provide a sub-domian address for their blogs. the sub-domain authority is exactly as the main domain. Whereas, for some other blog providers every subdomain has its different and lower authority. for example "ffff.blog.ir" and "blog.ir" both have domain authority of 60. It noteworthy to mention that the "ffff.blog.ir" does not even exist! This is while mihanblog.com and hfilm.mihanblog.com has diffrent page authority.
Intermediate & Advanced SEO | | rayatarh5451230 -
Domain dominance
I've just started to work for a company who've purchased masses of domains with every conceivable permutation based on all their products with every extension possible e.g .biz . eu. .net (including .co.uk and .com of course). I have two questions: 1. Is it worth keeping all these (they want to add more) domains or let them expire? 2. All the purchased domains are online - is there any point (they redirect with a 301)?
Intermediate & Advanced SEO | | LJHopkins0 -
Domain Redirect and SSL Cert
Hi, When redirecting an entire site to another domain, do you have to maintain the SSL certificate? The SSL expires 3 days before the planned redirect. Thanks in advance.
Intermediate & Advanced SEO | | sofla_seo0 -
How to do Country specific indexing ?
We are a business that operate in South East Asian countries and have medical professionals listed in Thailand, Philippines and Indonesia. When I go to Google Philippines and check I can see indexing of pages from all countries and no Philippines pages. Philippines is where we launched recently. How can I tell Google Philippines to give more priority to pages from Philippines and not from other countries Can someone help?
Intermediate & Advanced SEO | | ozil0 -
Using pictures from another domain
We are building several sites for several clients which will be using images from the manufacturer. Our dev team wants to insert the manufacturer's url for the images, instead of actually downloading the image and hosting on our server. There are thousands of images, so downloading images to our server will be time consuming, so we are looking for a shortcut.... however I'm concerned this will cause other issues. Is using manufactueresdomain.com/12345.jpg going to cause SEO issues? will this generate Google penalties? Since we are not able to control the image file name, we cannot optimize it. We will add Alt text and Title tag for each image, but the file name is random characters. How important is the file name for SEO?
Intermediate & Advanced SEO | | Branden_S0 -
Keep multiple domains or combine them?
I need some help figuring out if I should combine multiple domains or if I should let them be separate? I have domain1.com, domain2.com, and domain3.com. Well, domain1.com owns domain2.com and domain3.com. And currently domain1.com points to domain2.com and domain3.com from the homepage. They are going through some changes at their business, and now the option is on the table to combine the domains or still let them be separate as long as they link to each other. What is the best way to handle this and are there more things I should go through before making a decision? None of them have a ton of links to them, and they aren't super robust, but would just to have some advice. Thanks a lot
Intermediate & Advanced SEO | | Rocket.Fuel0 -
SEO value in multiple backlinks from same domain and from various sub-domains.
A site has a link to my site as one of their main tabs, which means whenever a user clicks through to another page within the site, my link - being a main tab - is there. This creates thousands of links from this site. How does Google treat this? Do we have a rough formula estimate. In other words, assume it creates 1,000 backlinks would the SEO value be around the same as if I had just 2 link total as a main tab, but on 2 different non-related sites? Or, does it actually count fully as 1,000 links? Links from various sub-domains. Several .EDU's are linking to my site. Different schools within the overall same university. Example: nursing.abc.edu links to my site, but so does business.abc.edu. For SEO does that count as much as if I had links from complete non-related universities, or would Google evaluate that these links are related (since same main domain) and that will discount any links more than 1 to some extent? If discounted, then what do we estimate the discount to be? thank yoyu
Intermediate & Advanced SEO | | knielsen1