Files blocked in robot.txt and seo
-
I use joomla and I have blocked the following in my robots.txt is there anything that is bad for seo ?
User-agent: *
Disallow: /administrator/
Disallow: /cache/
Disallow: /components/
Disallow: /images/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /libraries/
Disallow: /media/
Disallow: /modules/
Disallow: /plugins/
Disallow: /templates/
Disallow: /tmp/
Disallow: /xmlrpc/
Disallow: /mailto:myemail@myemail.com/
Disallow: /javascript:void(0)
Disallow: /.pdf
-
What you have there is just blocking rootdomain.com/javascript:void(0). Googlebot can execute and index JavaScript; you should not block it without a good reason. I'd let it read the JavaScript and see the submenus.
-
Thank you and is blocking javascript bad ? ( I was thinking about submenus )
-
If you don't want pages in those Disallowed directories to be indexed, then you're doing fine. These pages won't be able to be crawled, so, they won't be likely to appear in search results for any search engines.
The last three entries look fishy to me. I'd need to know what types of URLs you're trying to block to fix them. For the last one, if you're looking to block all pdfs on your site, the syntax would be Disallow: /*.pdf.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Regex in Disavow Files?
Hi, Will Regex expressions work in a disavow file? If i include website.com/* will that work or would you recommend just website.com? Thanks.
Intermediate & Advanced SEO | | Fubra0 -
Have a Robots.txt Issue
I have a robots.txt file error that is causing me loads of headaches and is making my website fall off the SE grid. on MOZ and other sites its saying that I blocked all websites from finding it. Could it be as simple as I created a new website and forgot to re-create a robots.txt file for the new site or it was trying to find the old one? I just created a new one. Google's website still shows in the search console that there are severe health issues found in the property and that it is the robots.txt is blocking important pages. Does this take time to refresh? Is there something I'm missing that someone here in the MOZ community could help me with?
Intermediate & Advanced SEO | | primemediaconsultants0 -
Where do we focus from an SEO perspective?
I run a digital business development consulting company. Our core has always been centered around consulting with client to develop strategy and then working with a set of vendors in a variety of different services types to implement the strategy. I'm struggling right now to determine the best approach for our own SEO. Our website is http://phase2solutions.net. What suggestions would you have on the approach here? Feel free to share your service as well as the tactical end is something we are looking to outsource. Thank you.
Intermediate & Advanced SEO | | phase2solutions0 -
Block lightbox content
I'm working on a new website with aggregator of content.
Intermediate & Advanced SEO | | JohnPalmer
i'll show to my users content from another website in my website in LIGHTBOX windows when they'll click on the title of the items. ** I don't have specific url for these items.
What is the best way to say for SE "Don't index these pages"?0 -
Domain Forwarding - SEO Impacts?
I have a site that has been active for years - thinkbiglearnsmart.com. Awhile ago I had purchased about 50 domain names that were relevant to my company. I still have those urls and would like to use them to point to different pages on my site - just because they have good key words in the URLs. For example - one is dreamweavertrainingclassesonlinelive.com. Currently they are all redirecting to my homepage. A. is that hurting me? B. I would like to redirect to the more relevant page. ie the page dedicated to Dreamweaver training (http://thinkbiglearnsmart.com/dreamweaver-creative-cloud-training-course/ ) Will this hurt my Dreamweaver keyword for example because there is already a 301 redirect on that page from a very old Dreamweaver link which was something like thinkbiglearnsmart.com/dreamweaver C. On my hosting account where I can select where the URL forwards to - it has an option for "Location forwarding" and "Frame forwarding" - currently they are set to Frame forwarding - which one is best? Any help is much appreciated!!! Thank you!
Intermediate & Advanced SEO | | webbmason0 -
How to lay off your SEO compnay?
I have decided to replace my seo company. The pint is this company has been partly my developer too. So he has set up a demo server of my website. 1- Should I be worried about duplicate material when I end my cooperation with this company(The demo server) 2- Should I be worried that if they do not like it, they go and delete all the submitted materials and destroy my pages rankings? Thanks all
Intermediate & Advanced SEO | | AlirezaHamidian0 -
Suggestions for a SEO Review
I think it would be beneficial to have a third party seo review of the network of sites my team and I manage and was wondering if any of you had suggestions for what sort of tests should be done or that we should expect to see done during one of these reviews. We are a small team who has varying seo experience and have been working hard to make improvements to our sites in the past year. Most of our sites have been completely overhauled in the last 12-16 months and seo work that had not been done in the past has been setup, along with some corrections that may have been harming seo. We believe we are on the right track and have learned a good amount about seo in that time, but it would be nice to have some "expert" feedback outside of our office to get a clearer picture on anything we may be missing or some suggested improvements. A sort of double check on the work we have done.
Intermediate & Advanced SEO | | unikey0 -
Social Media and SEO
What is best? Increasing DA and PA on a specific social media profile such as twitter or spreading out the DA and PA on a variety of different profiles?
Intermediate & Advanced SEO | | casper4340