Best way to block spambots in htaccess
-
I would like to block Russian Federation, China and Ukraine spam as well as semalt and buttonsforwebsite. I have come up with the following code, what do you think?
For the countries:
BLOCK COUNTRY DOMAINS
RewriteCond %{HTTP_REFERER} .(ru|cn|ua)(/|$) [NC]
RewriteRule .* - [F]And for buttons-for-website.com and semalt-semalt.com:
BLOCK REFERERS
RewriteCond %{HTTP_REFERER} (semalt|buttons) [NC]
RewriteRule .* - [F]or should it be:
BLOCK USER AGENTS
RewriteCond %{HTTP_USER_AGENT} (semalt|buttons) [NC]
RewriteRule .* - [F]Could I add (semalt|buttons|o-o-6-o-o|bestwebsitesawards|humanorightswatch) or is that too many?
-
Hi
I think you're on the right track.
A very good blog post by Jared Gardner has recently addressed this question on the Moz Blog.
Hope it helps,
Don
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What is the most effective way of selecting a top keyword per page on a site?
We are creating fresh content for outdated sites and I need to identify the most significant keyword per page for the content developers, What is the best way to do this?
Reporting & Analytics | | Sable_Group0 -
Is there a way to find out who the admin of a Google Analytics account is?
We have a client who has been searching high and low to gain access to the GA account that is tied to their website, but previous agencies keep stating they don't have access. Is there a way to find out who the admins are on a GA account with the tracking number so we can reach out to that person and ask them to admin the client?
Reporting & Analytics | | marketingmediamanagement0 -
Google Analytics and Bounce Rates Query - Should I block access from foreign countries ?
Hi , When I look at my google analytics for my UK Website, I can see alot of visits come from outside the UK , i.e Brazil and USA. Both of which give me almost 100% bounce rates from people visiting from there. I am wondering, if google looks at bounce rates with regards to ranking factors and should I therefore block access to my site from visitors outside the UK ?... Would this help increase my rankings ? Given that we only serve uk customers, I cant see any benefit of allowing non uk customers the ability to see the site . what does people think ? thanks pete
Reporting & Analytics | | PeteC121 -
What's the best way to figure out which keywords are the highest converting?
We have a client using Google Analytics. They currently have 3 goals set up to track when website visitors fill out 3 forms: Form A, Form B, Form C. I can easily figure out what traffic sources have driven the highest number of conversions on each form (Search for Form A, for instance, or Referrals for Form B), but of course, when I try to drill down on search terms that have driven conversions to each form, I get stuck in "not provided" territory. I'd like to know what people are searching for when they ultimately fill out each form. This will answer questions like: are people familiar with us already when they convert, or did they randomly find our website when searching for something we sell? It seems like there must be a way, using Google Webmaster Tools, Analytics, or another third-party app, to answer the question: what keyword searches are responsible for the highest number of conversions? Especially on a website that has traffic of 10,000+/month and a healthy dose of search traffic. Right? Where am I missing this information?
Reporting & Analytics | | timfrick1 -
Duplicate Title Errors on Product Category Pages - The best practice?
I'm getting quite a few 'Duplicate Title Error' on category pages which span over 2 - 3 pages. E.g. http://www.partwell.com/cutting-punches http://www.partwell.com/cutting-punches?page=1 http://www.partwell.com/cutting-punches?page=2 http://www.partwell.com/cutting-punches?page=3 All 4 pages currently have the same title... <title>Steel Cutting Punches</title> I was thinking of adding Page Numbers to the title of each corresponding page, thus making them all unique and clearing the Duplicate Page Title errors. E.g. <title>Steel Cutting Punches</title> <title>Steel Cutting Punches | Page 1 of 3</title> <title>Steel Cutting Punches | Page 2 of 3</title> <title>Steel Cutting Punches | Page 3 of 3</title> Is this the best way to go around it? Or is there another way that I'm not thinking of? Would I need to use the rel=canonical tag to show that the original page is the one I want to be found? Thanks
Reporting & Analytics | | bricktech0 -
Google Bot blocked
Google says that they are blocked from crawling my site. I have tried to get my web masters to look into this and they say the site apears fine. However, my SERP results have been falling and the errors are increasing. http://www.RealEstateMarketLeaders.com https://support.google.com/webmasters/bin/answer.py?hl=en&answer=2409441 I have sent an e mail into Google asking for help and I have been looking into every possible channel to find a solution for this problem. The site does have a 301 re direct pointing in from my old site http:///www.TriCitiesRealEstateAgent.com into my new site http://www.RealEstateMarketLeaders.com. However, I am not sure if that is the problem or if something else happend to make me see such a huge drop in SERP results.
Reporting & Analytics | | Brandon_Patton0 -
Blocking our IP's but wondering if Google still uses our search data?
The company owner here has our (company) website as his home page. I excluded our static IP’s on Google Analytics, but is that good enough to keep Google from using his search traffic as an indicator of anything negative. Does Google still take into account his activity, but simply block it from my reporting? Finally, does one person actually have that kind of influence as far as time on site, bounce rates, etc. Should I convince him to find a new home page?
Reporting & Analytics | | Ticket_King0 -
Search Engine blocked by robots.txt
I am getting this error whe I try to crawl http://photosales.belfasttelegraph.co.uk/ but my robots.txt file does not block any bots?
Reporting & Analytics | | MirandaP0