Do I need a separate robots.txt file for my shop subdomain?
-
Hello Mozzers!
Apologies if this question has been asked before, but I couldn't find an answer so here goes...
Currently I have one robots.txt file hosted at https://www.mysitename.org.uk/robots.txt
We host our shop on a separate subdomain https://shop.mysitename.org.uk
Do I need a separate robots.txt file for my subdomain? (Some Google searches are telling me yes and some no and I've become awfully confused!
-
Thank you. I want to disallow specific URLs on the subdomain and add the shop sitemap in the robots.txt file. So I'll go ahead and create another!
-
You go be fine without one. You only need one if you want to manage that subdmain: add specific xml sitemaps links in robots.txt, cut access to specific folders for that subdomain.
if you don't need any of that - just move forward without one.
-
Currently we just have: User-agent: *
I'm in the process of optimising.
-
It depends what currently is in your robots.txt. Usually it would be useful to have another one for your subdomain.
-
Yes, I would have a seperate robots.txt files.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Tags problem I need help
Hello I have a problem on my site and I want to help solve it I allow adding tags in search engines However, when writing new content, tags are not added to search engines This is my website link مجلة الحلوة
Technical SEO | | ahmedahmed3230 -
Robots User-agent Query
Am I correct in saying that the allow/disallow is only applied to msnbot_mobile? mobile robots file User-agent: Googlebot-Mobile User-agent: YahooSeeker/M1A1-R2D2 User-agent: MSNBOT_Mobile Allow: / Disallow: /1 Disallow: /2/ Disallow: /3 Disallow: /4/
Technical SEO | | ThomasHarvey1 -
Need Urgent Help
I have found one mistake that my place page address is little different than address on all local directories like on place page address is: 10010 S Tryon St #122 Charlotte, NC 28273 and on directories : 10010 South Tryon St 122 Charlotte, NC 28273 so on place page it is just "S" instead of South and "#" is before 122 but on all directories # is missing So what do you suggest ? Should i change address and re verify place page ? Re verify will put down place page value ???
Technical SEO | | mnkpso0 -
How many times robots.txt gets visited by crawlers, especially Google?
Hi, Do you know if there's any way to track how often robots.txt file has been crawled? I know we can check when is the latest downloaded from webmaster tool, but I actually want to know if they download every time crawlers visit any page on the site (e.g. hundreds of thousands of times every day), or less. thanks...
Technical SEO | | linklater0 -
No indexing url including query string with Robots txt
Dear all, how can I block url/pages with query strings like page.html?dir=asc&order=name with robots txt? Thanks!
Technical SEO | | HMK-NL0 -
Site Got Hacked! Need Help!
Hi Guys. One of my friend's site got hacked 2 weeks ago, because of bad php script hole and Google indexed the pages which got hacked and all the Title Tags and Descriptions are indexed in the Google which is very embarssing situation. All adult content texts. Right now we have solved the problem and closed the hole submitted the new sitemap, but Google is no longer coming back and refreshining the SERP. We have been waiting for 3 weeks for now? What should we do? Methods we tried so far: 1.Cleaned all meta tags generate new sitemap and submitted that to Google 2.Built some backlinks 3.Built some social bookmarks Thanks!
Technical SEO | | DigitalJungle0 -
Campaigns Domain and Subdomain... ?
I made two separate campaigns before I understood the meaning of "subdomain". I make one campaign for my www.com and another for my .com. I now realize I should have made the .com the domain and the www. the subdomain in the same campaign. Is there a way to edit this? Thanks!
Technical SEO | | musicforkids0 -
Robots.txt
Hi there, My question relates to the robots.txt file. This statement: /*/trackback Would this block domain.com/trackback and domain.com/fred/trackback ? Peter
Technical SEO | | PeterM220