Do I need a separate robots.txt file for my shop subdomain?
-
Hello Mozzers!
Apologies if this question has been asked before, but I couldn't find an answer so here goes...
Currently I have one robots.txt file hosted at https://www.mysitename.org.uk/robots.txt
We host our shop on a separate subdomain https://shop.mysitename.org.uk
Do I need a separate robots.txt file for my subdomain? (Some Google searches are telling me yes and some no and I've become awfully confused!
-
Thank you. I want to disallow specific URLs on the subdomain and add the shop sitemap in the robots.txt file. So I'll go ahead and create another!
-
You go be fine without one. You only need one if you want to manage that subdmain: add specific xml sitemaps links in robots.txt, cut access to specific folders for that subdomain.
if you don't need any of that - just move forward without one.
-
Currently we just have: User-agent: *
I'm in the process of optimising.
-
It depends what currently is in your robots.txt. Usually it would be useful to have another one for your subdomain.
-
Yes, I would have a seperate robots.txt files.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unknown Subdomains Ranking
In spot checking some pages that I recently launched, I found subdomains ranking in place of the domain. The strange thing is, we never set up these sub-domains and they don't exist on our server. The pages, though they're indexed with the proper title and meta-description, time out when clicked. We're operating on Drupal 7, pages launched at the beginning of the month. The other pages within the series of content are ranking properly. Any thoughts or tips to resolve this?
Technical SEO | | JordanNCU1 -
Do I have a robots.txt problem?
I have the little yellow exclamation point under my robots.txt fetch as you can see here- http://imgur.com/wuWdtvO This version shows no errors or warnings- http://imgur.com/uqbmbug Under the tester I can currently see the latest version. This site hasn't changed URLs recently, and we haven't made any changes to the robots.txt file for two years. This problem just started in the last month. Should I worry?
Technical SEO | | EcommerceSite0 -
Need help please with url guidelines.
Hi SEO PROS, I have a website and I am planing to change all the urls. I need to know what is the right way of making the urls. Here is some information. We are based in Brooklyn NY and we sell our services to Manhattan clients and Manhattan has few names. NY, NYC, Manhattan and NY. So by looking at my service area I came up with this url. http://www.signsny.com/brooklyn-ny/awnings this is my current url. http://www.signsny.com/sign-types/awnings-canopies-brooklyn-NYC. This is what I am planning to change it to. Please guide me to the right direction, so in future I don't have to re-do them again. Thanks Abie
Technical SEO | | signsny0 -
How to solve the meta : A description for this result is not available because this site's robots.txt. ?
Hi, I have many URL for commercialization that redirects 301 to an actual page of my companies' site. My URL provider say that the load for those request by bots are too much, they put robots text on the redirection server ! Strange or not? Now I have a this META description on all my URL captains that redirect 301 : A description for this result is not available because this site's robots.txt. If you have the perfect solutions could you share it with me ? Thank You.
Technical SEO | | Vale70 -
301 Redirect on a PDF, DOCX files?
Hi, I have to rename many pdf and docx files. How can I implement 301 redirect on them as they are linked from 'n' number of places? Regards, Shailendra Sial
Technical SEO | | IM_Learner1 -
New to rich snippets, help needed
Hi, I have an online store selling mens business attire in Australia. I have had my developers add the required code to allow rich snippets. You can see the result here http://www.google.com/webmasters/tools/richsnippets?url=http%3A%2F%2Fjsshirts.com.au%2Fmens-business-shirts%2Fclassic-fit%2Fsky-blue-poplin-classic-fit-shirt.html&view=cse I have few questions 1.How can I change the product description? Is it possible to use the product Meta Tag as the description? 2.Under the stars and the review count is this text. The excerpt from the page will show up here. The reason we can't show text from your webpage is because the text depends on the query the user types. Should whateever product description I have used show up here? 3.Is there anything else I need to do to get SERP's to show the snippet? Many thanks for any answers, Jason
Technical SEO | | mullsey0 -
Domains and subdomains
When I started a campaign for my message, I got the message: "We have detected that the domain www.vamospaella.com and the domain vamospaella.com both respond to web requests and do not redirect. Having two "twin" domains that both resolve forces them to battle for SERP positions, making your SEO efforts less effective. We suggest redirecting one, then entering the other here." I wasn't sure whether I had said it was a subdomain when in fact it was a domain (or the other way round), so I started another campaign for the same website using the other option and the message didn't come up. However, I still don't understand what you meant by this and whether it's an issue. When I search for my website in Google, it shows as vamospaella.com when other websites come up as www. and then their domain name. If it is a problem, is it to do with my hosting package and how it's set up or is it to do with my local site on my computer? I did ring my web host, 1&1, but they said they couldn't see a problem. Please can you let me know how I can resolve this as my ranking is still quite low in Google and I'm not sure why. If it is because of "twin domains", then will Google see my content as duplicated and keep me low in their rankings? I'm new to SEO and not a website novice, so please answer in lay terms! Thanks Melissa
Technical SEO | | melissa10 -
Robots.txt question
Hello, What does the following command mean - User-agent: * Allow: / Does it mean that we are blocking all spiders ? Is Allow supported in robots.txt ? Thanks
Technical SEO | | seoug_20050