Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
How do you de-index and prevent indexation of a whole domain?
-
I have parts of an online portal displaying in SERPs which it definitely shouldn't be. It's due to thoughtless developers but I need to have the whole portal's domain de-indexed and prevented from future indexing.
I'm not too tech savvy but how is this achieved? No index? Robots?
thanks
-
Fastest way to do it would be through a combination of Google Webmaster Tools and Robots.txt / noindex,nofollow meta tags.
Firstly if your domain hasn't been verified within GWT, go through that process. Then proceed with the remove URL request within GWT. (Image Attached)
You will need to upload a robots.txt and/or implement the noindex meta tag within the header of the pages you no longer want indexed.
Example Robots.txt:
User-agent: *
Disallow: /offendingportal/Example Robots Meta tag:
That should be enough to get rid of the offending content from within the SERPS and prevent it from being re-indexed.
-
Hi TCE,
There are two options in my opinion for this which are the best (as their are several other options including META robots etc):
- Robots.txt: Exclude your portal via your robots.txt file.
- HTTP Auth: Let your developers add HTTP Authentication so you have to log in to see the portal. Google won't be able to see the pages itself and so will de index them.
Hope this helps.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Does redirecting from a "bad" domain "infect" the new domain?
Hi all, So a complicated question that requires a little background. I bought unseenjapan.com to serve as a legitimate news site about a year ago. Social media and content growth has been good. Unfortunately, one thing I didn't realize when I bought this domain was that it used to be a porn site. I've managed to muck out some of the damage already - primarily, I got major vendors like Macafee and OpenDNS to remove the "porn" categorization, which has unblocked the site at most schools & locations w/ public wifi. The sticky bit, however, is Google. Google has the domain filtered under SafeSearch, which means we're losing - and will continue to lose - a ton of organic traffic. I'm trying to figure out how to deal with this, and appeal the decision. Unfortunately, Google's Reconsideration Request form currently doesn't work unless your site has an existing manual action against it (mine does not). I've also heard such requests, even if I did figure out how to make them, often just get ignored for months on end. Now, I have a back up plan. I've registered unseen-japan.com, and I could just move my domain over to the new domain if I can't get this issue resolved. It would allow me to be on a domain with a clean history while not having to change my brand. But if I do that, and I set up 301 redirects from the former domain, will it simply cause the new domain to be perceived as an "adult" domain by Google? I.e., will the former URL's bad reputation carry over to the new one? I haven't made a decision one way or the other yet, so any insights are appreciated.
Intermediate & Advanced SEO | | gaiaslastlaugh0 -
Google does not want to index my page
I have a site that is hundreds of page indexed on Google. But there is a page that I put in the footer section that Google seems does not like and are not indexing that page. I've tried submitting it to their index through google webmaster and it will appear on Google index but then after a few days it's gone again. Before that page had canonical meta to another page, but it is removed now.
Intermediate & Advanced SEO | | odihost0 -
Too many backlinks from one domain?
I've been in the process of creating a tourism-based website for the state of Kansas. I'm a photographer for the state, and have inked a nice little side income to my day job as a web designer by selling prints from Kansas (along with my travels elsewhere). I'm still in the process of developing it, but it's at least at a point that I need to really start thinking about SEO factor of the amount of backlinks I have from it going back to my main photography website. The Kansas site is at http://www.kansasisbeautiful.com and my photography website is http://www.mickeyshannon.com. This tourism website will serve a number of purposes: To promote the state and show people it's not just a flat, boring place. To help promote my photography. The entire site is powered by my photography. To sell a book I'm planning to publish later this year/early next year of Kansas images. To help increase sales of photography prints of my work. What I'm worried about is the amount of backlinks I have going from the Kansas site to my photography site. Not to mention every image is hosted on my photography domain (no need to upload to two domains when one can serve the same purpose). I'm currently linking back to my site on most pages via a little "Like the Photos? Buy a print" link in the top right corner. In addition, when users get to the website map, all photo listings click back to a page on my photography site that they can purchase prints. And the main navigation also has a link for "Photos" that takes them to my Kansas photo galleries on my photography website as well. The question I have: Is it really bad SEO-wise to have anywhere from 1 to 10+ backlinks on every page from one domain (kansasisbeautiful.com) linking back to mickeyshannon.com? Would I be better served moving all of the content from kansasisbeautiful into a subdirectory on my photography site (mickeyshannon.com/kansas/) and redirecting the entire domain there? I haven't actually launched this website yet, so I'm trying to make the right call before pushing it to the public. Any advice would be appreciated!
Intermediate & Advanced SEO | | msphoto0 -
Consolidating Multiple Domains into A Single Domain
I have a client who's website is an amalgamation of multiple domains. jacksonhole.net is the main domain but the site passes traffic back and forth from the following domains/sites. My questions is, would it it be better for SEO to consolidate all of these domains under the single high authority domain and 301 redirect the rest or is that a really bad idea? Thanks for your help. jacksonhole.net (Domain Authority 31) jackson-hole-rental-condos.com (Domain Authority 22) jackson-hole-rental-homes.com (Domain Authority 21) j acksonholehotelguide.com (Domain Authority 19)
Intermediate & Advanced SEO | | dbaxa-2613381 -
Community inside the domain or in a separate domain
Hi there, I work for an ecommerce company as an online marketing consultant. They make kitchenware, microware and so on. The are reviewing their overall strategy and as such they want to build up a community. Ideally, they would want to have the community in a separate domain. This domain wouldn't have the logo of the brand. This community wouldn't promote the brand itself. The brand would post content occassionally and link the store domain. The reasoning of this approach is to not interfere in the way of the community users and also the fact that the branded traffic acquired doesn't end up buying at the store I like this approach but I am concerned because the brand is not that big to have two domains separated and lose all the authority associated with one strong domain. I would definitely have everything under the same domain, store and community, otherwise we would have to acquire traffic for two domains. 1. What do you think of both scenarios, one domain versus two? Which one is better? 2. Do you know any examples of ecommerce companies with successful communities within the store domain? Thanks and regards
Intermediate & Advanced SEO | | footd0 -
Redirecting index.html to the root
Hi, I was wondering if there is a safe way to consolidate link juice on a single version of a home page. I find incoming links to my site that link to both mysite.com/ and mysite.com/index.html. I've decided to go with mysite.com/ as my main and only URL for the site and now I'd like to transfer all link juice from mysite.com/index.html to mysite.com/
Intermediate & Advanced SEO | | romanbond
When i tried 301 redirect from index.html to the root it created an indefinite loop, of course. I know I can use a RewriteRule.., but will it transfer the juice?? Please help!7 -
Keep multiple domains or combine them?
I need some help figuring out if I should combine multiple domains or if I should let them be separate? I have domain1.com, domain2.com, and domain3.com. Well, domain1.com owns domain2.com and domain3.com. And currently domain1.com points to domain2.com and domain3.com from the homepage. They are going through some changes at their business, and now the option is on the table to combine the domains or still let them be separate as long as they link to each other. What is the best way to handle this and are there more things I should go through before making a decision? None of them have a ton of links to them, and they aren't super robust, but would just to have some advice. Thanks a lot
Intermediate & Advanced SEO | | Rocket.Fuel0 -
Indexed Pages in Google, How do I find Out?
Is there a way to get a list of pages that google has indexed? Is there some software that can do this? I do not have access to webmaster tools, so hoping there is another way to do this. Would be great if I could also see if the indexed page is a 404 or other Thanks for your help, sorry if its basic question 😞
Intermediate & Advanced SEO | | JohnPeters0