Identifying Bots in Google Analytics
-
Hi there,
While you can now filter out bots and spiders in Google Analytics, I'm interested in how you identify a bots and spiders in the first place. For example, it used to be thought that Googlebot wouldn't appear in GA as it 'couldn't process Javascript' but now Google has announced new developments for its crawler with regards to interpreting javascript and CSS, this argument isn't as cut and dry.
I'm not suggesting Googlebot appears in Google Analytics, but I am saying that you can't make the case that it won't appear only because it can't interpret JavaScript.
So, I'm interested to see what metrics you use to identify a bot? For me, the mix of Users > Browser, Users > Operating System Version is still quite handy, but is it possible to identify individual bots and spiders within Google Analytics? And would Googlebot appear?
-
Hi there,
That's great - thanks. I'm aware that GA can block bots data from appearing within the settings (I think I kind of flagged this in the first sentence of my question, but probably could have been clearer), but I was a bit more interested in identifying potential bots and spiders within GA; e.g are there any noticeable tells within the browser version/operating system data that can suggest bot behaviour.
For example, if you segment a browser version, operating system and then filter only for pageviews where time on page ≤ 00:10 or less (for example) can you start to isolate bot behaviour in GA?
Edit: or are the numbers so small (due to the amount of bots that can execute javascript) that there wouldn't be much data?
-
Hi,
By default Google Analytics blocks all the well know bots and spiders that are out there and I would say the do a pretty good job. As enabling the feature to disable even more bots and spiders made less than 1% of visitors disappear with over millions of visitors.
They're able to identify bots and spiders based on the user agents so that's why they can also easily block them.
-
Hello.
I recommend reading the following articles:
http://www.lunametrics.com/blog/2014/08/07/bot-spider-filtering-google-analytics/
In short, there is a checkbox under Admin settings. > _View _panel>"View Settings." But eitherway read about it to see if it is worth it for you.
They are very recent and give a lot of insight on the subject.
Adidtionaly, is your site using wordpress or a similar CMS? If so, there are a couple of pligins that ell you when a bot has visited your site.
Thanks and hope it helps
Ricardo
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to set goal in Google Analytics that required specific page
So our company has new page that has just implemented (let say "page x" --> not a landing page) and we want to see how many visitors that through "page x " convert into the goal (let say "page y"). If I just make the goal destination like "/page y" the goal number that appear is ALL the visitors who reach "page y" (through or not through "page x"), so how I set the goal setting to only show the visitors who reach "page y" through "page x" ? Thank you
Reporting & Analytics | | ddspg0 -
Google Analytics - Sessions vs Views?
In Google Analytics under Audience Overview I want to visit total visits during any given period of time but instead it shows me sessions. Am I missing something? How do I switch from sessions to page visits?
Reporting & Analytics | | gohawks77902 -
Is it possible to use Google Tag Manager to pass a user’s text input into a form field to Google analytics?
Hey Everyone, I finally figured out how to use auto event tracking with Google Tag Manager, but didn't get the data I wanted. I want to see what users are typing into the search field on my site (the URL structure of my site isn't set up properly to use GA's built-in site search tracking). So, I set up the form submit event tracking in Google Tag Manager and used the following as my event tracking parameters: Category: Search Action: Search Value When I test and look in Google Analytics I just see: "search" and "search value." I wanted to see the text that I searched on my site. Not just the Action and Category of the event.... Is what I'm trying to do even possible? Do I need to set up a different event tracking parameter? Thanks everyone!
Reporting & Analytics | | DaveGuyMan0 -
Referral Traffic vs. Campaign Traffic in Google Analytics
I have two sites: a blog and an ecommerce site. The blog funnels people to the ecommerce site. In Analytics I'm seeing declines in referral traffic from the blog to the ecommerce site. During the same time I'm seeing an increase in campaign traffic to the ecommerce site, with most campaign traffic coming from the blog. I believe the increase in campaign traffic is largely a result of simply having installed more tracking links. This leads me to believe that the declines I'm seeing in referral traffic is simply a result of the increase in campaign traffic. In other words, what was once counted and reported as being referral traffic is now being counted and reported as campaign traffic. So my question is this: In Google Analytics is campaign traffic ALSO reported as referral traffic, or is campaign traffic reported separately and not duplicated in referral traffic reports? I'll provide a concrete example to make this more clear in case it isn't: Say site X sends 1000 visits each month to site Y. Say 50 of those visits come from a single link on X. If that link is changed so that campaign Z data info added (via the Google URL Builder), would you expect to then see 950 referral visits each month from site X to site Y plus 50 campaign visits to site Y via new campaign Z, or would you continue to see 1000 referral visits plus the new 50 campaign visits? Many thanks in advance to anyone that can shed some light on this.
Reporting & Analytics | | aaronprimal0 -
Google Analytics set up for non-canonicalized domains
Our client's website is non-canonicalized (www.example.com & example.com load the same thing). Google seems to have made a preference for the www, but canonicalizing to www breaks their Flash website. All we're really trying to do at this time is install Google Analytics for them. What's the smartest way to make sure that both www.example.com and example.com are treated exactly the same by Google Analytics? Google Developers: Domains & Directories states that by default visit data will be separately collected between the two domains, although I found no references to the common www/naked domain issue. In stackoverflow: Does google analytics combine naked domains with the www subdomain? Török Gábor says, "Yes, users will be tracked, but the same visitor coming from www.datalookups.com and datalookups.com will be counted as two different visitors." On the same page, Open SEO says, "This is completely false: www.domain.tld and domain.tld are treaded just the same, and get the same value for the HASH code (the number at the start of each __utm cookie). This an exception: every other subdomain.domain.tld will be handeld as a distinct web site". Can any Analytics experts help me sort this out? Thanks!
Reporting & Analytics | | GOODSIR0 -
Track banner ad with google analytics
Hi We have a top banner ad on an external renown site only to attrack traffic. It does show up in google analytics, but i have seen some increase in (direct) / (none) traffic. Is there anyway i can track it when banner is showed from 2 separate URLs http://www.tv2torget.no/bedrift/netthandel/ and http://www.tv2torget.no/bedrift/netthandel/barneklaer/ Thanks Dan Lærum
Reporting & Analytics | | danlae0 -
Can you get local search numbers/traffic out of Google Analytics?
With Google's new local search I am more curious as to market penetration on keywords that are now localized to my different US cities. I understand that you can separate out Google traffic based on regional Google domains, but I am curious if there is an effective way to separate out searches and keywords based on a my local US Metros? If google cannot do this, any recommendations on products that can? Thanks.
Reporting & Analytics | | Thos0031 -
Best Practices For Google Analytics
Does anyone have any recommendations or resources for best practices in using Google Analytics. Specifically, as they relate to SEO and managing and monitoring several websites (i.e. registering/managing multiple accounts vs. master account, etc).
Reporting & Analytics | | Gyi0