Identifying Bots in Google Analytics
-
Hi there,
While you can now filter out bots and spiders in Google Analytics, I'm interested in how you identify a bots and spiders in the first place. For example, it used to be thought that Googlebot wouldn't appear in GA as it 'couldn't process Javascript' but now Google has announced new developments for its crawler with regards to interpreting javascript and CSS, this argument isn't as cut and dry.
I'm not suggesting Googlebot appears in Google Analytics, but I am saying that you can't make the case that it won't appear only because it can't interpret JavaScript.
So, I'm interested to see what metrics you use to identify a bot? For me, the mix of Users > Browser, Users > Operating System Version is still quite handy, but is it possible to identify individual bots and spiders within Google Analytics? And would Googlebot appear?
-
Hi there,
That's great - thanks. I'm aware that GA can block bots data from appearing within the settings (I think I kind of flagged this in the first sentence of my question, but probably could have been clearer), but I was a bit more interested in identifying potential bots and spiders within GA; e.g are there any noticeable tells within the browser version/operating system data that can suggest bot behaviour.
For example, if you segment a browser version, operating system and then filter only for pageviews where time on page ≤ 00:10 or less (for example) can you start to isolate bot behaviour in GA?
Edit: or are the numbers so small (due to the amount of bots that can execute javascript) that there wouldn't be much data?
-
Hi,
By default Google Analytics blocks all the well know bots and spiders that are out there and I would say the do a pretty good job. As enabling the feature to disable even more bots and spiders made less than 1% of visitors disappear with over millions of visitors.
They're able to identify bots and spiders based on the user agents so that's why they can also easily block them.
-
Hello.
I recommend reading the following articles:
http://www.lunametrics.com/blog/2014/08/07/bot-spider-filtering-google-analytics/
In short, there is a checkbox under Admin settings. > _View _panel>"View Settings." But eitherway read about it to see if it is worth it for you.
They are very recent and give a lot of insight on the subject.
Adidtionaly, is your site using wordpress or a similar CMS? If so, there are a couple of pligins that ell you when a bot has visited your site.
Thanks and hope it helps
Ricardo
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
In google analytic for google /cpc it is showing url with 404 which even not exists in my database
Hello All, In google analytic for google /cpc it is showing url with 404 which even not exists in my database that also more than 300 per day. How can it is possible? it is showing /black-friday-offers but I don't have such page. Thanks!
Reporting & Analytics | | pragnesh96390 -
How do you analyze a traffic drop with no historic Google Analytics data?
A client of mine has a large website with multiple sections (shop, forums, articles, etc.) that apparently had a significant reduction in rankings, traffic, and sales in the past. However, historic Google Analytics data is not available for the site, and I'm having troubles identifying anything concrete about the traffic drop, such as when it happened, what pages/sections it happened to, etc. The shop traffic drives most of the revenue, but it's a small number compared to the forums traffic, so it's hard to pick anything out of top-line trends like SEMrush offers. What tools or strategies might help in this situation?
Reporting & Analytics | | AdamThompson0 -
Conflicting data in Google Analytics
Hi Guys I've been looking at the data for a client in Google analytics and I was wondering if anyone knows why some of the data doesn't tally up. In my case its the following: Under Aquisition
Reporting & Analytics | | Relative
No of Sessions for a Keyword shown in Organic Search tab (compared to)
No of Clicks for a Query in the Search Engine Optimisation tab For example, for a brand term, Google are showing 17 Sessions in Organic Search.
For the same term Google are showing 90 Clicks in the Queries section of Search Engine Optimisation OK, we know that Google are a little cloak and dagger regarding keyword data but surely Sessions and Clicks for the same keyword should be identical unless I'm missing something.0 -
Google Analytics (not provided)
Hi A greater and greater % of clicks is showing as "not provided" - does any one know of any good external tracking software which will show this information? Thanks
Reporting & Analytics | | jj34340 -
Google Analytics traffic hijacking?
Ran into something interesting a week ago - the same Google Analytics code was installed on two different sites by accident. The account was reporting traffic from both domains. Haven't found a definitive answer on how to stop this yet if it were to be used maliciously?
Reporting & Analytics | | khemistry0 -
Google Analytics: Do url redirects show up
Good Morning from 6 degrees c mostly cloudy wetherby UK 🙂 A redirect has been set up with the effect that when you key in www.hyload.co.uk it forwards to www.ikogroup.co.uk My question is please...within Google analytics will referall traffic from www.hyload.co.uk show up in referral traffic or because its a redirect will it be counted as direct traffic. Thanks in advance, David
Reporting & Analytics | | Nightwing0 -
Exporting Keywords in Google Analytics
In the old Google Analytics, I would simply add &limit=30000 in the URL. Does anyone know how to export all keywords from the new Google Analytics?
Reporting & Analytics | | TommySwanson520 -
Does Google Analytics parse visits from search apps?
Does anyone know if Google Analytics reports visits to your website differently from individual search apps like Google and Bing? Or do they just treat them the same as any other keyword visit from Google or Bing search engine? I suppose the end result is probably the same as in the Google app you're using Google so it would just be a different access point versus a new tool. I'm just curious if there is a way to see how many Mobile visits are coming from the apps vs the browser. For me personally I have the Google and Bing iOS apps installed but rarely use them, opting for the Safari search bar 99% of the time.
Reporting & Analytics | | nsauser0