Files blocked in robot.txt and seo
-
I use joomla and I have blocked the following in my robots.txt is there anything that is bad for seo ?
User-agent: *
Disallow: /administrator/
Disallow: /cache/
Disallow: /components/
Disallow: /images/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /libraries/
Disallow: /media/
Disallow: /modules/
Disallow: /plugins/
Disallow: /templates/
Disallow: /tmp/
Disallow: /xmlrpc/
Disallow: /mailto:myemail@myemail.com/
Disallow: /javascript:void(0)
Disallow: /.pdf
-
What you have there is just blocking rootdomain.com/javascript:void(0). Googlebot can execute and index JavaScript; you should not block it without a good reason. I'd let it read the JavaScript and see the submenus.
-
Thank you and is blocking javascript bad ? ( I was thinking about submenus )
-
If you don't want pages in those Disallowed directories to be indexed, then you're doing fine. These pages won't be able to be crawled, so, they won't be likely to appear in search results for any search engines.
The last three entries look fishy to me. I'd need to know what types of URLs you're trying to block to fix them. For the last one, if you're looking to block all pdfs on your site, the syntax would be Disallow: /*.pdf.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Is robots met tag a more reliable than robots.txt at preventing indexing by Google?
What's your experience of using robots meta tag v robots.txt when it comes to a stand alone solution to prevent Google indexing? I am pretty sure robots meta tag is more reliable - going on own experiences, I have never experience any probs with robots meta tags but plenty with robots.txt as a stand alone solution. Thanks in advance, Luke
Intermediate & Advanced SEO | | McTaggart1 -
SEO question
Hi there! I'm the SEO manager for 5 Star Loans. I have 2 city pages running. We are running our business in 2 locations: Berkeley, CA & San Jose, CA. For those offices we've created 2 google listings with separate gmail accounts. Berkeley (http://5starloans.com/berkeley/) ranks well in Berkeley in Gmaps and it shows on first page in organic results. However the second city page San Jose (http://5starloans.com/san-jose/) doesn't show in the Gmaps local pack results and also doesn't rank well in organic results. Both of them have authentic backlinks and reviews. It has been a year already and it's high time we knew the problem 🙂 any comment would be helpful. thanks a lot
Intermediate & Advanced SEO | | moonalev0 -
Robots.txt vs noindex
I recently started working on a site that has thousands of member pages that are currently robots.txt'd out. Most pages of the site have 1 to 6 links to these member pages, accumulating into what I regard as something of link juice cul-d-sac. The pages themselves have little to no unique content or other relevant search play and for other reasons still want them kept out of search. Wouldn't it be better to "noindex, follow" these pages and remove the robots.txt block from this url type? At least that way Google could crawl these pages and pass the link juice on to still other pages vs flushing it into a black hole. BTW, the site is currently dealing with a hit from Panda 4.0 last month. Thanks! Best... Darcy
Intermediate & Advanced SEO | | 945010 -
Massive URL blockage by robots.txt
Hello people, In May there has been a dramatic increase in blocked URLs by robots.txt, even though we don't have so many URLs or crawl errors. You can view the attachment to see how it went up. The thing is the company hasn't touched the text file since 2012. What might be causing the problem? Can this result any penalties? Can indexation be lowered because of this? ?di=1113766463681
Intermediate & Advanced SEO | | moneywise_test0 -
Do these results indicate a problem with my seo?
I've entered my the following search query into Google.co.uk related:mywebsite.co.uk However the resulting website that are brought back are on the whole nothing like our website, nor do they offer similar services to us. If I run this same query on my competitors websites they all bring back similar websites to each other. I read somewhere that gaining links from the websites that Google believes are similar/related to our own website is beneficial. But looking at our results it would seem that Google can't place what our site is about and which sites are similar. So I'm guessing this is a more pressing matter than link building right now!? Other info about our website: We rank fairly well for a lot of our target keywords.
Intermediate & Advanced SEO | | adamlcasey
Domain age = 11 years
PA =38
mR= 4.77
mT= 5.74
DA:= 31
DmR= 3.78
DmT= 3.84
PageRank = 3 Example of how random the results are the 1st website that comes back in our related websites search is for Doctors GP Practice. Our website sells GPS Telematics Solutions. Can anyone shed any light on this or just to confirm how much of a problem this is?0 -
SEO for bigcommerce site
I have a site on bigcommerce platform .from Where do i need start SEO for these types of ecommerce sites.Looking for Experts ideas . Thank you.
Intermediate & Advanced SEO | | innofidelity0 -
SEO Provider
At the risk of opening the flood gates, I'm posting this in hopes of finding a SEO provider on this site. I'm currently a member, and have learned a lot through this site, but I'm still having issues ranking higher. Because of this, I have come to the conclusion that I need a dedicated SEO company/professional who can help our company reach the next level. I'm open to suggestions of companies that you have used, or please feel free to throw your hat into the ring. Thanks in advance! AW
Intermediate & Advanced SEO | | TheCTC0 -
SEO Consultant for site audit
Can someone recommend an excellent SEO who can perform a full site audit of my fairly large Wordpress site? The site receives about 14,000 visits per month but traffic is waining one month after a recent change. Need analysis of some funky stuff in my Webmaster tools and overall site review.
Intermediate & Advanced SEO | | JSOC0