One robots.txt file for multiple sites?

[renalynd27]

I have 2 sites hosted with Blue Host and was told to put the robots.txt in the root folder and just use the one robots.txt for both sites. Is this right? It seems wrong. I want to block certain things on one site.

Thanks for the help,

Rena

[RyanPurkey]

Hi Rena. Yes, if both sites are separate domains that you want to use in different ways, then you should place a different robots.txt file in each domain root so that they're accessible at xyz.com/robots.txt and abc.com/robots.txt.  Cheers!

[LoganRay]

Hi Rena,

You technically can do that, but it's not recommended - for the exact reason you state above. More often than not, 2 sites aren't going to have the same set of disallow rules.

Additionally, you should also be using robots.txt files to direct search engines to your XML sitemap, and if you're sharing a robots file, then you can't specify 2 different sitemaps on 2 different domains.

[GlobeRunner]

Each individual website (and some subdomains if you add them) needs a unique robots.txt file. You can copy the same file and use it again and again on each site, but each one needs a robots.txt file.