Identifying Bots in Google Analytics
-
Hi there,
While you can now filter out bots and spiders in Google Analytics, I'm interested in how you identify a bots and spiders in the first place. For example, it used to be thought that Googlebot wouldn't appear in GA as it 'couldn't process Javascript' but now Google has announced new developments for its crawler with regards to interpreting javascript and CSS, this argument isn't as cut and dry.
I'm not suggesting Googlebot appears in Google Analytics, but I am saying that you can't make the case that it won't appear only because it can't interpret JavaScript.
So, I'm interested to see what metrics you use to identify a bot? For me, the mix of Users > Browser, Users > Operating System Version is still quite handy, but is it possible to identify individual bots and spiders within Google Analytics? And would Googlebot appear?
-
Hi there,
That's great - thanks. I'm aware that GA can block bots data from appearing within the settings (I think I kind of flagged this in the first sentence of my question, but probably could have been clearer), but I was a bit more interested in identifying potential bots and spiders within GA; e.g are there any noticeable tells within the browser version/operating system data that can suggest bot behaviour.
For example, if you segment a browser version, operating system and then filter only for pageviews where time on page ≤ 00:10 or less (for example) can you start to isolate bot behaviour in GA?
Edit: or are the numbers so small (due to the amount of bots that can execute javascript) that there wouldn't be much data?
-
Hi,
By default Google Analytics blocks all the well know bots and spiders that are out there and I would say the do a pretty good job. As enabling the feature to disable even more bots and spiders made less than 1% of visitors disappear with over millions of visitors.
They're able to identify bots and spiders based on the user agents so that's why they can also easily block them.
-
Hello.
I recommend reading the following articles:
http://www.lunametrics.com/blog/2014/08/07/bot-spider-filtering-google-analytics/
In short, there is a checkbox under Admin settings. > _View _panel>"View Settings." But eitherway read about it to see if it is worth it for you.
They are very recent and give a lot of insight on the subject.
Adidtionaly, is your site using wordpress or a similar CMS? If so, there are a couple of pligins that ell you when a bot has visited your site.
Thanks and hope it helps
Ricardo
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Analytics Numbers Are Weird
Hi, I'm having a strange problem from past one month. My website gets about 10k pageviews a month with an average bounce rate of 50%. Lately, I observed a strange thing. The avg time on page of visitors from selected countries is less than 1 second and the bounce rate is zero!! How is this possible? This is happening from past one month and it would be really helpful if you guys could tell me what exactly is happening?? Ive attached a screenshot for better understanding. website url : https://www.specbee.com/ fIAXXgF 3zQuBAN
Reporting & Analytics | | ganesh10 -
Google Analytics, new property or views?
Hi All We have an existing Google analytics account which was customised relatively heavily a few years back, with custom variables, e-commerce tracking and the a large number of filters. We have determined that the custom variables and e-commerce tracking, at least for now, are no longer required and actually complicating our lives. We want to simplify the account and add a fresh set of goals and events. Our initial train of thought was to create an entire new property with the usual views, raw, test, main etc. We are proposing to retain both properties (6 months) so we can still do some comparative stuff. We're now wondering if we should retain the existing property and just create new views, the downside being that whilst we can compare data with previous periods it's meaningless to some degree as the filters etc are different. The plus of more views is that we're not firing the tag twice retaining two properties. Views and opinions welcome. Thanks
Reporting & Analytics | | mde9110 -
Why is Google Analytics reporting 20% fewer goals than Unique pageviews of same thank you page?
This is really puzzling me and my research has not thrown out the answer. I have always understood URL goals to be unique pageviews of the thank you page you are tracking. UPVs and goals should both only be counted once per session... Has anyone else seen this issue? Goals were not set up historically so I wanted to use unique pageviews of the thank you page for year on year comparisons, but 20% is a big difference! Background There are multiple pages to track so goal is set up using Regex There is no mistake in the goal set up (honest!) The goal URLs all match the unique pageview URLs, there are no rogue URLs There has been no change to the site or the tracking set up Data is not being sampled It's a lead gen site in an area where multiple enquiries within one visit would be very unusual Thanks in advance!
Reporting & Analytics | | McCannSEO0 -
Google Analytics Report throws up Google as a referrer
Good morning from Wet & Windy 12 degrees C wetherby UK... Using Google analytics I've noticed in the traffic sources refferer subsection some traffic is categorized as originating from Google. Whats puzzling me is.... I know a huge amount of traddic stems from Google but as the below screenshot illustrates only 21 visitors come from Google: http://i216.photobucket.com/albums/cc53/zymurgy_bucket/google-refferal-sources-top-levelcopy.jpg And when i drill down some are coming from Google mobile 😞 http://i216.photobucket.com/albums/cc53/zymurgy_bucket/google-referral-sourcescopy.jpg Is traffic categorised as Google referrer down to Google hiding searches via ssl as explained here: http://i216.photobucket.com/albums/cc53/zymurgy_bucket/google-referral-sourcescopy.jpg Any insights welcome 🙂
Reporting & Analytics | | Nightwing0 -
Google Analytics - Referral Traffic Question
How Google Analytics determine that some particular web site referred traffic if there is no back link on that site?
Reporting & Analytics | | DiamondJewelryEmpire1 -
Google Analytics internal Site Search - Destination pages dispaly Search results
Hi, Im having a bit of an issue with Google Analytics internal site search, I am able to currently track the search terms through my website internal search but when I click onto destination pages I just get the search result page. When clicking destination pages I would expect to get the pages on which the user ended up after the results page, instead I just get the results page which is pretty much useless ?submitsearchXXXXXX hope you can help, look forward to your response. Thanks,
Reporting & Analytics | | Tug-Agency1 -
Disclaimer about using Google Analytic?
When using GA on my Website do I need to have a disclaimer that 'We are using Google Analtyic to Track information'?
Reporting & Analytics | | daracreative0 -
Google Analytics: how many visits from country Google domains?
Hello, I manage a site with visitors from many different countries. With Google Analytics, it is normal to see the number of visitors from each search engine. However, I would like to identify the number of visitors from each Google-search contry domain. How many visitors from Google.com? How many from Google.co.uk. And from Google.co.zm? And so on. Anybody knows if this is possible and if yes, how can it be done? Thank you in advance, Dario
Reporting & Analytics | | Darioz0