Whatstuffwherebot user agent messing up Google Analytics
-
Starting yesterday, Aug 26, 2020, I noticed a new bot crawling our site with user agent whatstuffwherebot.
Google Analytics is counting these hits as human traffic, completely throwing off my numbers - yesterday, Analytics reported nearly triple my typical number of visitors.
As of now, Search Console only shows data through Aug 25 so I don't know if Search Console is also affected.
Is anybody else seeing something similar?
Does anybody know what the whatstuffwherebot bot is? I don't get any results when I search on Google or Bing.
For what it's worth, the traffic is coming from Columbus, OH, running over Amazon AWS via 278 different IP addresses so far. Also, WordFence (my WordPress security plugin) correctly identifies these hits as bot traffic.
-
Hi Martijn,
Thanks for the reply.
Yes, I've enabled bot filtering on all views except my unfiltered view. My site gets so much bot traffic that the data would be unusable without it.
As for filtering the traffic, I'm not aware of any way to retroactively filter out User-Agent unless I use GTM to create a custom dimension as described here: https://www.addsearch.com/docs/analytics/addsearch-bot-google-analytics/
Perhaps this is something I should do in case something like this ever happens again in the future.
I'm still hoping for other suggestions - particularly ways to ask the Google Analytics team to include "whatstuffwherebot" as part of their bot filtering.Best,
Akira -
Hi Akira,
Have you enabled bot filtering as a setting in Google Analytics on a view level? If not, you could always add a filter on the property or account level to filter out the traffic if it has very common patterns as you're describing. Usually, I would just stay away, from filtering out this type of traffic though if you have large amounts of traffic as otherwise you might be doing more harm to your data long-term than helping it.
Martijn.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Important updates on Google Analytics Data Retention and the General Data Protection Regulation (GDPR)
Hi Everyone, I'm sure many of you received the email from Google over the past few days with the subject line: [Action Required] Important updates on Google Analytics Data Retention and the General Data Protection Regulation (GDPR). I hope I'm not alone in not knowing what exactly this whole notification was in regards to. I realize it's for Data but are we no longer able to pull stats from the past? If anyone has a "dumbed down" explanation for what this update entails, I would be very interested - I don't want to miss out on any important updates and info, but I'm just not grasping this content. Below is the full email in its entirety for those who are interested as well: Dear Google Analytics Administrator,
Reporting & Analytics | | MainstreamMktg
Over the past year we've shared how we are preparing to meet the requirements of the GDPR, the new data protection law coming into force on May 25, 2018. Today we are sharing more about important product changes that may impact your Google Analytics data, and other updates in preparation for the GDPR. This e-mail requires your attention and action even if your users are not based in the European Economic Area (EEA).
Product Updates
Today we introduced granular data retention controls that allow you to manage how long your user and event data is held on our servers. Starting May 25, 2018, user and event data will be retained according to these settings; Google Analytics will automatically delete user and event data that is older than the retention period you select. Note that these settings will not affect reports based on aggregated data.
Action: Please review these data retention settings and modify as needed.
Before May 25, we will also introduce a new user deletion tool that allows you to manage the deletion of all data associated with an individual user (e.g. site visitor) from your Google Analytics and/or Analytics 360 properties. This new automated tool will work based on any of the common identifiers sent to Analytics Client ID (i.e. standard Google Analytics first party cookie), User ID (if enabled), or App Instance ID (if using Google Analytics for Firebase). Details will be available on our Developers site shortly.
As always, we remain committed to providing ways to safeguard your data. Google Analytics and Analytics 360 will continue to offer a number of other features and policies around data collection, use, and retention to assist you in safeguarding your data. For example, features for customizable cookie settings, privacy controls, data sharing settings, data deletion on account termination, and IP anonymization may prove useful as you evaluate the impact of the GDPR for your company’s unique situation and Analytics implementation.
Contract And User Consent Related Updates
Contract changes
Google has been rolling out updates to our contractual terms for many products since last August, reflecting Google’s status as either data processor or data controller under the new law (see full classification of our Ads products). The new GDPR terms will supplement your current contract with Google and will come into force on May 25, 2018.
In both Google Analytics and Analytics 360, Google operates as a processor of personal data that is handled in the service.
• For Google Analytics clients based outside the EEA and all Analytics 360 customers, updated data processing terms are available for your review/acceptance in your accounts (Admin ➝ Account Settings).
• For Google Analytics clients based in the EEA, updated data processing terms have already been included in your terms.
• If you don’t contract with Google for your use of our measurement products, you should seek advice from the parties with whom you contract.
Updated EU User Consent Policy
Per our advertising features policy, both Google Analytics and Analytics 360 customers using advertising features must comply with Google’s EU User Consent Policy. Google's EU User Consent Policy is being updated to reflect new legal requirements of the GDPR. It sets out your responsibilities for making disclosures to, and obtaining consent from, end users of your sites and apps in the EEA.
Action: Even if you are not based in the EEA, please consider together with your legal department or advisors, whether your business will be in scope of the GDPR when using Google Analytics and Analytics 360 and review/accept the updated data processing terms as well as define your path for compliance with the EU User Consent Policy.
Find Out More
You can refer to privacy.google.com/businesses to learn more about Google’s data privacy policies and approach, as well as view our data processing terms.
We will continue to share further information on our plans in the coming weeks and will update relevant developer and help center documentation where necessary.
Thanks,
The Google Analytics Team6 -
How to track in google analytic which filters selected in which category?
Hi All, Can anyone tell me how to track which filters selected in which category like in category I have 1) price ( then different - different price) 2) color ( then difference - difference colors) same for size, width, brand etc. Is it possible to track in detail via tag manager? Other than tag manager what are the best way to track filters? Please explain me technical guidance too. Thanks!
Reporting & Analytics | | adamjack0 -
Google Analytics for User Experience
Hi In terms of looking at the overall User Experience of a website is there any particular areas of Google analytics that you believe to be particularly useful to identify areas of worry or opportunity?
Reporting & Analytics | | TheZenAgency1 -
Mobile Text message about Intelligence Alerts (Google Analytics)
Mobile text message service for Custom Alerts that set up under Analytics Intelligence is available in all countries worldwide? Is it True or False? Can you please explain me..i am confuse about this answer.
Reporting & Analytics | | CommercePundit0 -
Funnel Visualization in Google Analytics
Hello. I have a question about funnel visualization in Google Analytics where there is an eCommerce goal (i.e. add to cart, shipping, purchase) with the first step being mandatory. Issue:
Reporting & Analytics | | Unity
The amount of completed purchases reported in GA doesn't match the actual transaction figure via the eCommerce platform. Question:
I was wondering if anyone knows why this occurs? I have a few ideas on why however I haven't been able to locate any supporting evidence. So I was hoping to generate a few ideas from the SEOMoz community to help me investigate further. Any ideas welcome! Much thanks,
Davinia0 -
What is s.ytimg.com in google analytics?
My clients GA reports 273 visits from s.ytimg.com. I go to the site, it doesn't exist. I googled it, there were some code with s.ytimg.com in it, but nothing I could understand. Anybody have an idea where this comes from?
Reporting & Analytics | | endlessrange0 -
Omniture vs Google Analytics
What's your opinion on analytics tools? Specifically Omniture vs Google Analytics. Is Omniture really that much more powerful than GA? Have you used GA Premium, the enterprise package? My main question - Can GA (free or premium) do everything that Omniture can? If GA Premium and Omniture were the same price, which would you choose? Is one harder to implement, pull reports, tweak, ect? Thanks in advance.
Reporting & Analytics | | akim260