Whatstuffwherebot user agent messing up Google Analytics
-
Starting yesterday, Aug 26, 2020, I noticed a new bot crawling our site with user agent whatstuffwherebot.
Google Analytics is counting these hits as human traffic, completely throwing off my numbers - yesterday, Analytics reported nearly triple my typical number of visitors.
As of now, Search Console only shows data through Aug 25 so I don't know if Search Console is also affected.
Is anybody else seeing something similar?
Does anybody know what the whatstuffwherebot bot is? I don't get any results when I search on Google or Bing.
For what it's worth, the traffic is coming from Columbus, OH, running over Amazon AWS via 278 different IP addresses so far. Also, WordFence (my WordPress security plugin) correctly identifies these hits as bot traffic.
-
Hi Martijn,
Thanks for the reply.
Yes, I've enabled bot filtering on all views except my unfiltered view. My site gets so much bot traffic that the data would be unusable without it.
As for filtering the traffic, I'm not aware of any way to retroactively filter out User-Agent unless I use GTM to create a custom dimension as described here: https://www.addsearch.com/docs/analytics/addsearch-bot-google-analytics/
Perhaps this is something I should do in case something like this ever happens again in the future.
I'm still hoping for other suggestions - particularly ways to ask the Google Analytics team to include "whatstuffwherebot" as part of their bot filtering.Best,
Akira -
Hi Akira,
Have you enabled bot filtering as a setting in Google Analytics on a view level? If not, you could always add a filter on the property or account level to filter out the traffic if it has very common patterns as you're describing. Usually, I would just stay away, from filtering out this type of traffic though if you have large amounts of traffic as otherwise you might be doing more harm to your data long-term than helping it.
Martijn.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to change domains in Google Analytics without losing the data
Hi there, We recently changed our domain from .COM to .NET so that all our subdomains from external pages matched. Right now in Google Console we have our new .NET website being tracked, but in GA we are still tracking .COM. It is also causing issues with MOZ crawling our site because of the .COM/.NET discrepancy. My question is what is the best way to change our Google Analytics from .COM to .NET without losing historical data and what considerations do we need to change before implementing this? Our team was concerned that just downloading the old data would be too vast and it we wouldn't be able to continue manipulating it dynamically in GA. Thanks!!
Reporting & Analytics | | cPanel-LLC.0 -
When Deployment done for my site in google analytic page views little bit affected
Hello Expert, For my ecommerce site every minute I have 500 visitors now when my development team do deployment like minor JS update where site restart not require then what I found is on website (front end) not able to find anything happen that means visitor not event realize page even refresh but in analytic real time it shows minor pageviews down. So can anyone guess here what happen actually in this case as per google? Thanks!
Reporting & Analytics | | micey1230 -
Identifying Bots in Google Analytics
Hi there, While you can now filter out bots and spiders in Google Analytics, I'm interested in how you identify a bots and spiders in the first place. For example, it used to be thought that Googlebot wouldn't appear in GA as it 'couldn't process Javascript' but now Google has announced new developments for its crawler with regards to interpreting javascript and CSS, this argument isn't as cut and dry. I'm not suggesting Googlebot appears in Google Analytics, but I am saying that you can't make the case that it won't appear only because it can't interpret JavaScript. So, I'm interested to see what metrics you use to identify a bot? For me, the mix of Users > Browser, Users > Operating System Version is still quite handy, but is it possible to identify individual bots and spiders within Google Analytics? And would Googlebot appear?
Reporting & Analytics | | ecommercebc0 -
Google Analytics: Okay to change domain?
So, we are a long time user of GA and we're planning a domain change.
Reporting & Analytics | | jmueller0823
Does anyone know if I can 'change the domain' in GA so we don't lose our past data?
Thanks!0 -
Google analytics and software applications
Hei Guys. I think i know the answer for this one but i thought i ask you in order to be 100% sure. Ok let's go.. So i set up url based goals in Google analytics. My website (what are running on WordPress) has google analytics enabled but just before customers makes desired action i have to send them to the application page. Trick is that the application page is not running on wordpress and doesn't have google analytic tracking. After customer fills the application form i send him to my /thank-you page on my wordpress site. My question is: Does the conversion still count because customer left my website for a minute in order to fill in the application form? Best Regards, Tauri
Reporting & Analytics | | seopartnermarketing0 -
How to remove unwanted dynamic parameters from a URL in Google Analytics
Hi, Would really appreciate some help with this. I have been experimenting with RegEx to achieve this but as I’ve never used it before am currently failing miserably. We have conversion pages i need to set goals for that are formatted as below: https://www.domain.co.uk//Application_Form/(S(ewhbqp5cki0mppuzukunkqno))/enterCardDetails.aspx I need to remove the (s(xxx)) section from the URL as rather than one pages i currently have thousands of unique URL's. What’s catching me out is that as it’s not a URL parameter I can’t discount and as half way through can’t just do head matches etc to /entercarddetails Help would be much appreciated. Thanks.
Reporting & Analytics | | Sarbs0 -
Is there a way to use Google Analytics event tracking for YouTube embedded videos on my website?
Hello! I am trying to track different types of interaction on the YouTube for videos which are embedded on a website to measure the interaction on these video from my organic traffic. Is there a good way to go about this with the code since these are coming from an iFrame on YouTube? Would appreciate any feedback or help on implementing the event tracking with YouTube videos. Many thanks!
Reporting & Analytics | | CabbageTree0 -
Setting up Google Analytic Goals to a 3rd Party Site
I recently received help on a question I asked on SEOmoz but need additional clarification. I am trying to set up goals in Google Analytics for people who click on a “purchase botton” which sends them to PayPal. I created a Thank You page and tried to get PayPal to redirect to it, however, our customers only get to our site’s 404 page. Here is what I’ve done so far: Went into my PayPal account and turned the “Auto Return” to ‘on’ Under website payment preferences, I added the following URL http://www.teecycle.org/thank-youutm_nooverride1. (I formatted the URL this way because the person who provided me with help recommended using the format ?UTM_nooverride=1. However, our CMS system won’t allow “?” or “=”)
Reporting & Analytics | | EricVallee340