KW Explorer is Working to Disambiguate Keywords Google Merges Together
-
Hey gang,
Russ Jones from Moz has been doing a ton of heavy lifting work to try to get around the new problem posed by Google AdWords recent change to merged-keyword volume data. But, we're fighting back against this obfuscation in Keyword Explorer. I'm sharing two emails (slightly edited) from Russ about what we're doing here:
Introduction to the Problem:
Google Adwords Keyword Planner is the primary source for keyword search volume (how often a keyword is searched monthly on Google) for much of the search marketing industry. While Google has grouped together highly-similar terms for a while (especially misspellings), in June of 2016 they dramatically increased this keyword-grouping. This means similar phrases like "keyword rank", "keyword ranking" and "keyword rankings" would all be reported as having the same, combined search volume, rather than their individual search volumes. If you were to take Google's numbers at face value, you might think there are 3,000 searches per month for these 3 terms, when in reality there is only 1,000, divided amongst the 3 terms.
How we are addressing it:
Moz's Keyword Explorer uses a blend of data sources, not just Keyword Planner, to build our volume metrics. This gives us a distinct advantage in that we can adjust the volume of words that deviate dramatically in one data set verses another. Take for example the phrases "keyword rank", "keyword ranking", and "keyword rankings". While Google Keyword Planner might report all of these as having 1,000 searches per month, Moz Keyword Explorer can detect that these numbers are significantly higher than what our models would predict given our other data sets. We can then adjust the volume accordingly. Moreover, given our huge keyword data set, we can also identify grouped phrases (like these 3) and divide the volume proportionally to what we see in our other data sets. Thus, we address the grouping problem from multiple directions.
Here's email #2 from Russ, detailing more of how we're attacking this:
I have been working pretty much non-stop on this keyword volume disambiguation problem (finding the real search volume of individual keywords when Google clumps several together). I think I have settled on a pretty good solution and am working on getting it all in. For example...
Google Keyword Volume for the phrases "briefcase for women" and "briefcases for women" are both at 3600 because they have been lumped together. My disambiguation script says the singular (briefcase for women) should be 2731 and the plural should be 869. Google Trends roughly agrees with this, showing that the singular is searched more than 2x the plural: https://www.google.com/trends/explore#q=briefcase%20for%20women%2C%20briefcases%20for%20women&cmpt=q&tz=Etc%2FGMT%2B4
Basically, Keyword Explorer should already be providing some more accurate/segmented numbers than AdWords, and in the future, we'll get even better thanks to our clickstream data and our evolving models.
Any questions, let us know!
-
Thank you, Russ!
Google confirmed me that this change is due to an update, it won't be rolled back and there's no way to fetch data for each variant individually. Details are here.
Your tool will help us a lot.
-
Thanks for taking the time to come up with a 'solution' to the problem
-
The bug has been fixed but it will take 24 hours for the caches to clear! Thanks again!
-
That is a weird bug, because our database is showing 18k+ for that term! I will investigate!
-
Thank you, Rand and Russ!
I found a weird result. "bag" in US is reported to have 0-10, while "bags" in US is 11.5k-30.3k.
Considering KW planner and Google trend, "bag" in US seems to be outlier.
-
You're working in the right direction, guys. It seems a little scammy on the part of search engines to show these search volumes, as if they prompt you to bid on those keywords because they have a high search volume. While they do show the same SERP for this keyword, I believe a page optimized with "Buy a car in Orlando" will have a better score and a lower bid than a page optimized for "Buys a cars in Orlandos".
-
Thanks!
-
You're welcome for our efforts. I am really excited as we roll out this volume fix. I think it is going to offer a real improvement for SEOs everywhere.
-
Thanks Russ and Rand! I've used many tools for getting to the bottom of estimated volume and have high hopes for your evolving KW Explorer. It's definitely needed.
-
This has a HUGE HISTORICAL impact.
-
Thanks for touching upon this topic which is seriously impacting all marketers who rely on google data. Even the core definition of "Search Volume" is now NOT correct. It's THAT bad as pointed out here http://bit.ly/29MvvvK. Thanks for your efforts.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
On Page Grade & Page Optimization Tools not working
Hello! I have an issue with my on page grader and Page optimization. Over the past two weeks i get the attached errors. I searched the forums but after contacting our developer I am in a bit of a standstill. Could someone help on what triggers this? Would highly appreciate it. Best,
Moz Bar | | pranbg
Pran c5CSsHK c5CSsHK0 -
Why do i get a difficulty and CTR score in keyword explorer when there is no min or max monthly volume on a search term?
why do i get a difficulty and CTR score in keyword explorer when there is no min or max monthly volume on a search term?
Moz Bar | | eezysee0 -
Ranking now showing on google
So. its kind of weird. on my campaign keywords tracking I now have 4 keywords that my client site is being ranked for. (on Monday it was 2) however on Monday and now too. I cannot find my website when I try and search that keyword manually. i understand that Moz does not update and search every keyword every day but even after a few days now. also the rank tracker show different results for same keywords and these results too seems not accurate what am i missing?
Moz Bar | | SharonEKG0 -
What is a Good Keyword Organic CTR Score?
Hi Folks! You might have seen my discussion on What Is a Good Keyword Difficulty Score, and this is a continuation of the same vein. Keyword Organic CTR is probably my favorite score we developed in Keyword Explorer and Moz Pro. It looks at the SERP features that appear in a set of results (e.g. an image block, AdWords ads, a featured snippet, or knowledge graph) and then calculates, using CTRs we built off our partnership with Jumpshot's clickstream data, what percent of searchers are likely to click on the organic, web results. For example, in a search query like Nuoc Cham Ingredients, you've got a featured snippet and then a "People Also Ask" feature above the web results, and thus, Keyword Explorer is giving me an Organic CTR Score of 64. This translates directly to an estimated 64% click-through rate to the web results. Compare that to a search query like Fabric Printed Off Grain, where there's a single SERP feature - just the "People Also Ask" box, and it's between the 6th and 7th result. In this case, Keyword Explorer shows an Organic CTR Score of 94, because we estimate that those PAAs are only taking 6% of the available clicks. There are two smart ways you should be using Organic CTR Score: As a way to modify the estimated volume and estimated value of ranking in the web results for a given keyword term/phrase (KW Explorer does this for you if you use the "Lists" and sort based on Potential, which factors in all the other scores, including volume, difficulty, and organic CTR) As a way to identify SEO opportunities outside the normal, organic web results in other SERP features (e.g. in the Nuoc Cham Ingredients SERPs, there's serious opportunity to take over that featured snippet and get some great traffic) OK, so all that said, what's actually a "good" Organic CTR score? Well... If you're doing classic, 10-blue-links style SEO only, 100 is what you want. But, if you're optimizing for SERP features, and you appear in a featured snippet or the image block or top stories or any of those others, you'd probably be very happy to find that CTR was going to those non-web-results sections, and scores in the 40s or 50s would be great (so long as you appear in the right features).
Moz Bar | | randfish12 -
How non-US Moz customers will use Keyword Explorer after the Keyword Difficulty tool is retired?
The new Moz Keyword Explorer looks good but its search volume is US based and completely useless for non-US websites. This is from Rand's post: "while the tool can search any Google domain in any country, the volume numbers will always be for US-volume. In the future, we hope to add volume data for other geos as well." In the Keyword Difficulty tool, Moz shows Google search volume data, which is similar to what I see in the Google Keyword Planner and Google Search Console. For example, keyword X in the Australian search market has 6-7k searches in the Google Keyword Planner and 8k searches in Moz. The very same keyword has 118k-300k search volume in the new Keyword Explorer! Obviously this new search volume is not useful in the Australian market. I often used the Keyword Difficulty tool to identify new keyword opportunities but what can I do to complete the same tasks after they retire the tool?
Moz Bar | | Gyorgy.B2 -
In Moz Medium Pro how many pages can you track for keyword rankings at a time? I know you can track up to 50 in Standard.
I would possibly like to upgrade my subscription from Standard to Medium because I want to be able to track more pages in Rank Tracker. Unfortunately, the product descriptions do not include how many pages you can track in Rank Tracker.
Moz Bar | | VacasaSEM0 -
Does MozPro provide Google Avg. Monthly Search Counts for keywords?
Moz Pro provides Bing Local and Global search counts for specific keywords. Does it provide keyword specific search counts for Google results? If it does, how to I find them? Thank you! Dan
Moz Bar | | SearchOpt0 -
How to print segmented keyword lists to PDF?
Hi Moz forum, We're trying to print segmented keyword lists to PDF. Using the 'segmented lists' feature, we have created segmented lists for several different keyword groupings. We are looking to print each list individually as a PDF, but the feature seems to print ALL keywords from the entire campaign. How can we print segmented keyword lists to PDF? Thanks
Moz Bar | | BoomDialogue690