I am looking for best way to block a domain from getting indexed ?
-
We have a website http://www.example.co.uk/ which leads to another domain (https://online.example.co.uk/) when a user clicks,in this case let us assume it to be Apply now button on my website page. We are getting meta data issues in crawler errors from this (https://online.example.co.uk/) domain as we are not targeting any meta content on this particular domain. So we are looking to block this domain from getting indexed to clear this errors & does this effect SERP's of this domain (**https://online.example.co.uk/) **if we use no index tag on this domain.
-
Hi the real challenge here is we are not using any Google entities like webmaster tools etc on https://online.example.co.uk so to my knowledge robots.txt wont work, will be waiting to hear from you if we have any other options here.
Thanks
P
-
I'd recommend putting a robots.txt file on the https://online.example.co.uk site you don't want indexed.
Just save the following as robots.txt:
User-agent: *
Disallow: /then add it to the the root folder of the site like https://online.example.co.uk/robots.txt
This tells all the search spiders not to index or crawl any pages on the entire site.
-
Hi thanks for letting me know this , it will be great if you have any wildcard to solve this
-
If you use the meta noindex on page then it will be blocked from the index. So, yes, it will effect the SERPs of that domain by removing any results from the SERPs.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Why My site pages getting video index viewport issue?
Hello, I have been publishing a good number of blogs on my site Flooring Flow. Though, there's been an error of the video viewport on some of my articles. I have tried fixing it but the error is still showing in Google Search Console. Can anyone help me fix it out?
Technical SEO | | mitty270 -
Root domain change - how do we best handle existing backlinks from our own content platforms on youtube, etc?
Hi, we have recently changed our brand name after 7 years and have changed our root domain to match (33shake.com since 2012, now 33 fuel.com) The site is the same (no migration to a new one) as there were no other business changes apart from the name/domain. 301 redirects are looking after all former 33shake.com links, which are now being redirected to their new 33fuel.com equivalents (slugs are the same in 99% of cases). My question is: We have a lot of backlinks for our old domain (33shake.com) on our own content via our YouTube channel (100+ videos) and also our podcast (64 episodes in, broadcast on 10 platforms). For maximum SEO benefit as we continue to restore domain authority, etc to 33fuel.com, are we best to leave these historical backlinks pointed at the old domain and let the redirects pick them up when people click? Or are we better off swapping all of these old historical backlinks so they point directly to the new domain? Any advice would be greatly appreciated, this is quite a maze we are now picking our way through! Warren
Technical SEO | | WP331 -
Best way to change URL for already ranking pages
Hello. I have a lot of pages that I'm optimising. The ones I'm focusing on right now is already ranking, but the URLs could be better (they don't include the keywords right now). However I'm worried that if I change the URLs they will drop in rankings or have to start over. I would of course set up 301 redirect, but is there more I need to do? What is the best way to change URL for already ranking pages?
Technical SEO | | GoMentor0 -
Why am I getting millions of links from my root domain to my subdomains?
My site's subdomains (us.example.com, de.example.com, etc.) are showing millions of links (in Google Webmaster Tools) from the root domain. This seems very unnatural to me. Any idea what would be cause this or is this? In addition, I just found out that we deliberately stop googlebot crawling GEO-IP redirects, so that when googlebot tries to crawl our UK, DE, FR, etc. sites, it is not redirected to us.example.com. I'm thinking they may be linked? Thanks for your help!
Technical SEO | | CMcC0 -
301 from old domain to new domain
Hi, I need to create a 301 redirect for all internal pages located on organic7thheaven.com to the homepage of our new site at http://www.7thheavennaturals.com/ Currently internal pages of the old site such as the following are returning a page not found www.organic7thheaven.com/products/deepcleansing/miraclemud.asp Can anyone help me in setting up a .htaccess file for this problem please? Thanks
Technical SEO | | MJMarketing0 -
Warnings for blocked by blocked by meta-robots/meta robots Nofollow...how to resolve?
Hello, I see hundreds of notices for blocked by meta-robots/meta robots nofollow and it appears it is linked to the comments on my site which I assume I would not want to be crawled. Is this the case and these notices are actually a positive thing? Please advise how to clear them up if these notices can be potentially harmful for my SEO. Thanks, Talia
Technical SEO | | M80Marketing0 -
What to do with domain?
I bought a domain a couple months ago while having delusions of granger. The domain is PR2, has around 30 decent backlinks, and a small amount of traffic. It also has one of our keywords in the domain name. I'm thinking of 301 redirect to our homepage. Is there a downside to doing this? We really could use the additional traffic on our site.
Technical SEO | | dmac0