I am looking for best way to block a domain from getting indexed ?
-
We have a website http://www.example.co.uk/ which leads to another domain (https://online.example.co.uk/) when a user clicks,in this case let us assume it to be Apply now button on my website page. We are getting meta data issues in crawler errors from this (https://online.example.co.uk/) domain as we are not targeting any meta content on this particular domain. So we are looking to block this domain from getting indexed to clear this errors & does this effect SERP's of this domain (**https://online.example.co.uk/) **if we use no index tag on this domain.
-
Hi the real challenge here is we are not using any Google entities like webmaster tools etc on https://online.example.co.uk so to my knowledge robots.txt wont work, will be waiting to hear from you if we have any other options here.
Thanks
P
-
I'd recommend putting a robots.txt file on the https://online.example.co.uk site you don't want indexed.
Just save the following as robots.txt:
User-agent: *
Disallow: /then add it to the the root folder of the site like https://online.example.co.uk/robots.txt
This tells all the search spiders not to index or crawl any pages on the entire site.
-
Hi thanks for letting me know this , it will be great if you have any wildcard to solve this
-
If you use the meta noindex on page then it will be blocked from the index. So, yes, it will effect the SERPs of that domain by removing any results from the SERPs.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Multisite domain
good morning I have a wordpress site I have activated the multisite, currently the site has a domain authority of 8, when I publish a post, it is indexed quite quickly, if I publish a post in a language other than the /es subdomain it takes 24 hours why? If the author domain is the same, why does the employee take longer to be indexed on Google? Thank you
Technical SEO | | alainscilly770 -
Website is not indexing
Hi All, My website URL is https://thepeopeople.com and it is neither caching nor indexing in Google. Earlier the URL was https://peopeople.com. I have redirected it to https://thepeopeople.com by using 301 redirections. I have checked the redirection and everything else is fine and I have submitted all the URLs in search console also, still the website is not indexing. Its been more than 5 months now. Please suggest a solution for this. Thanks in Advance.
Technical SEO | | ResultfirstGA0 -
Changed domains, saw significant drop in domain authority
Hello, A little over a month ago we switched our domain from A to B. The original was .com and the new one is at a .tech for good measure. We have meticulously done 301 redirects page by page but there has been a massive drop in traffic and I checked here to see that we've gone from a 30 domain authority to a 9. The company is a few months over 1 year old, but we're really looking at traffic accumulated in just about 8 months for the old domain. Is there any way to recover some of the old juice in this one other than the re-directs? Our number of backlinks have also severely dropped despite re-directs.
Technical SEO | | SteveSaf0 -
301 Domain Redirect And Old Domain to a New one including pages
Hi, I need to 301 an old domain to a new one (new website) I need to 301 the domain to a new page not the new domain direct for example www.olddomain.co.uk to www.newdomain.co.uk/pagenew Then I need to also 301 all the other pages on the old domain to the new one for example... www.oldmain.co.uk/oldpage to www.newdomain.co.uk/newpage Issue is I can do one or the other not both, I can get the other pages to redirect but then the main domain wont redirect to the correct new page. Or I can get the old domain to redirect but not the internal pages. Thanks
Technical SEO | | David-Sharpe0 -
What is the best way to show content in Listing Pages?
If it is e-commerce site and a product listing page there is always a conflict how to show the content? As per my understanding we can show content in two different ways. 1. To show little content and use **Read more. (**In this case there is a direct message to the google: Here is the content visible and rest content is hidden but available for visitors to read more 2. Can use** Scroll bar**. So here is the message to Google and visitors that my full content is available here. So just scroll down to read further. So I want to know that which method of showing content is best and it's impact of SEO where there is UI constraint or both the method is ok without any SEO impact. Please share your suggestions. DCdRJpH
Technical SEO | | kathiravan0 -
How should I close my forum in a way that's best for SEO?
Hi Guys, I have a forum on a subdomain and it is no longer used. (like forum.mywebsite.com) It kind of feels like a dead limb and I don't know what's best to do for SEO. Should I just leave it as it is and let it stagnate? There is a link in the nav menu to the main domain so users have a chance to find the main domain. Or should I remove it and just redirect the whole subdomain to the main domain? I don't know if redirects would work as I doubt most of the threads would match our articles, plus there are 700 of them. The main domain is PR3 and so is the forum subdomain. Please help!
Technical SEO | | HCHQ0 -
I have a blog on a sub domain, would you move it to the rood domain in a directory?
I have a blog that preforms fairly well on a sub domain, but after reading a post that Rand made to the Q & A I am thinking about moving it to the main domain in a sub directory. What are your thoughts on this? Here are some stats on it. The blog currently gets about 5 x the traffic of the main domain. The domain is older, 2008 creation date. They pretty much register for the same keywords.
Technical SEO | | LesleyPaone0 -
What's the best way to deal with an entire existing site moving from http to https?
I have a client that just switched their entire site from the standard unsecure (http) to secure (https) because of over-zealous compliance issues for protecting personal information in the health care realm. They currently have the server setup to 302 redirect from the http version of a URL to the https version. My first inclination was to have them simply update that to a 301 and be done with it, but I'd prefer not to have to 301 every URL on the site. I know that putting a rel="canonical" tag on every page that refers to the http version of the URL is a best practice (http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=139394), but should I leave the 302 redirects or update them to 301's. Something seems off to me about the search engines visiting an http page, getting 301 redirected to an https page and then being told by the canonical tag that it's actually the URL they were just 301 redirected from.
Technical SEO | | JasonCooper0