Can Robots.txt on Root Domain override a Robots.txt on a Sub Domain?
-
We currently have beta sites on sub-domains of our own domain. We have had issues where people forget to change the Robots.txt and these non-relevant beta sites get indexed by search engines (nightmare).
We are going to move all of these beta sites to a new domain that we disallow all in the root of the domain.
If we put fully configured Robots.txt on these sub-domains (that are ready to go live and open for crawling by the search engines) is there a way for the Robots.txt in the root domain to override the Robots.txt in these sub-domains?
Apologies if this is unclear. I know we can handle this relatively easy by changing the Robots.txt in the sub-domain on going live but due to a few instances where people have forgotten I want to reduce the chance of human error!
Cheers,
Dave.
-
Hi Dave. A workflow checklist should really help with this as well. There are probably a few other items you'll catch by meeting with the others involved and getting everyone on the same page. Cheers!
-
Dave, I had exactly the same issue a month ago with being indexed on subdomains but was able to modify robots.txt in root domain swiftly enough to avoid real damage. Your main root robots.txt can override the subdomains. Simply disallow the subdomain in your robots file disallow: /subdomain url/robots.txt OR, if im not mistaken simply remove it altogether from the subdomain
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
If Product Pages Perform Well In Google then Is It Possible That Category Page Can perform Well In Google?
Hi All, For my ecommerce site I have optimized my product pages very nicely like good images, detailed information about products, good reviews, implemented schema for my product and reviews and very perfect onpage. Now my query is if my products pages performing well in google then there are chances that my category page rank well in google too? Thanks!
On-Page Optimization | | wright3350 -
Where can we see voice search impressions/searches?
Hi Guys, As voice search seems to be getting more popular, I would like to start accounting for these searches. I've tried looking for low impressions/long tail searches in the Search Console but feel like I am just guessing which queries are search. Does anybody have any clever way of finding these searches or could you list an example of how/if you search using voice? Thanks!
On-Page Optimization | | Xtend-Life0 -
Can Javascript Links Be Used to Reduce Links per Page?
We are trying to reduce the number of links per page, so for the low-value footer links we are considering coding them as javascript links. We realize Google can read java, but the goal is to reduce level of importance assigned to those internal links. Would this be a valid approach? So the question is would converting low-value footer links to js like below help reduce the number of links per page in google's eyes even though we're reasonably sure they can read javascript. <a <span="" class="html-tag">href</a><a <span="" class="html-tag">="</a><a class="html-attribute-value html-external-link" target="_blank">javascript:void(0);</a>" data-footer-link="/about/about">About Us
On-Page Optimization | | Jay-T0 -
Can't get my preferred URL, how much does it matter?
Hi guys. I'm building a new site at the moment - seen a solid SEO opportunity for my work. I'm a producer engineer, specialising in mixing and mastering, so i'm creating a site for online mixing services. After a bit of keyword research I decided that "online mixing" was the best, most relevant and high volume term to go for. Ideally i'd like my home page to be www.onlinemixing.com (or something similar) but alas! It's been taken, as well as all the variations (like switching words, hypens etc) How much does this matter form an SEO point of view? E.g - For the search term "online mixing" would - www.onlinemixing-signalchain.co.uk be much worse than -www.onlinemixing.co.uk? Or am I sweating the small stuff? Any thoughts would be greatly appreciated. Isaac.
On-Page Optimization | | isaac6630 -
Can I use the same text in my meta description as I put in my post excerpt?
Hi, I'm just trying to understand the right way to optimise my blog posts and this is likely a dumb question... but to what extent should the text in my meta description differ from the text in my post excerpts? cheers, Andrew
On-Page Optimization | | seowhiskey0 -
Can a page lose ranking because it has too many bold tags?
I run a product search website for the Engineering Sector. Each of the companies listed on the website has a profile page. This page used to rank very well in google. Over the last couple of weeks these pages disappeared for google i.e they fell about 20 to 30 spots in the search results for relevant keywords like the Manufacturer Name. I was trying to analyze the page and found that Each of the Categories that the manufacturer supports was listed in "bold" and linked to the page for that category. A manufacturer can support up to 20 or 30 categories so this results in about 20 or 30 Bold keywords. Is this a bad practice? Could this be a reason for a drop in ranking or do bold keywords on a page not matter? What are your views?
On-Page Optimization | | raghavkapur0 -
Schema business address - can this affect me in the national rankings
Looking at using Schema microdata on our new website, we currently rank quite decently in Google UK. If I was to add schema mark up for our business address could this effect us in Google UK rankings wise i.e. our rankings for national search (for want of a better word!) suffer but our local rankings increase?
On-Page Optimization | | Jon-C0