Thoughts on scraping SERPs and APIs
-
I'm perplexed about a couple of things.
Google says that scraping keyword rankings is against their policy from what I've read. Bummer. We comprise a lot of reports and manual finding and entry was a pain. Enter Moz! We still manually check and compare, but it's nice having that tool. I'm confused now though about practices and getting SERPs in an automated way. Here are my questions
- Is it against policy to get SERPs from an automated method? If that is the case, isn't Moz breaking this policy with it's awesome keyword tracker?
- If it's not, and we wanted to grab that kind of data, how would we do it? Right now, Moz's API doesn't offer this data. I thought Raven Tools at one point offered this, but they don't now from what I've read. Are there any APIs out there that we can grab this data and do what we want with it? (let's day build our own dashboard)?
Thanks for any clarification and input!
-
You're much too kind
Happy New Year!
-
Hi Rand. Anyone ever tell you you're awesome?
I just did. Thanks bud
-
Hi Boogily,
A) In terms of ToS, I'm not 100% clear. I know that Google has been strict about automated rank checking programs installed on a user's local machine, and they've enforced that a bit through technical and guidelines means. However, they've been unclear about companies like Moz, Searchmetrics, GetStat, AuthorityLabs, Conductor, Brightedge, Yield, RankAbove, and hundreds of others which provide this data in their software. I suspect they don't love it, but tolerate it and haven't taken action for over a decade now.
B) On the API front, check with GetStat and AuthorityLabs. I believe they both have rank data offerings from an API standpoint. Moz doesn't offer that and probably won't in the future, either.
Cheers!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What is the difference between my pages' keyword rank and keyword-specific SERP?
I've been tasked with identifying high volume keywords for products our company sells and optimizing our copy and listings to rank higher on Google. The Keyword Explorer and On-Page Grader have been fantastic for this, but as I write my reports, i'm not sure whether SERP or Moz's keyword page rank would be more accurate for reporting results and movement. For instance, one of our product pages ranks #12 for the highest (relevant) volume keyword, but when looking at the SERP analysis for that keyword, our product page actually shows up at #7 down the list. I've tried naturally searching in Google in case it was an issue of private results based on my user data but it seems to give the same result. Am I conflating two different tools? Or is the Moz rank just an estimate?
Moz Pro | | BJake0 -
Can a search query return more than one SERP feature for a single keyword?
In Moz's 'SERP Features' section within campaigns, it shows each of my tracked keywords as being eligible for multiple SERP features - sometimes as many as 5 or 6. Does this mean that Google could display a few different rich snippets for these keywords, or just that it could return one of those highlighted?
Moz Pro | | rwat0 -
SERPs ans SEO
I am trying to analyze my site to determine what we may be doing wrong as far as our site goes Ilike everyone else I suppose). In any event I have followed as much as I can from the Moz guideline and tools. We were ranking very well for quite some time and now..kaput. We have tried to make our site faster (using Google Page Speed ..the CDM for GOOGLE). We have also tried redoing out title and description tags to be more search and user friendly. Our site is very fast now...which is good. But we are still not seeing the rankings improvements that we had hoped for. We have built the site using WP and broke the template to make what we feel is an attractive site....used what we feel are the best plugins to capitalize....can anyone give us some feedback to let us know if we are just missing something here. We are RELEVANT to our industry...
Moz Pro | | AdsOnWheels0 -
Are Moz Ratings that reliable/correlated against SERPS?
So 3 weeks in Penguin 2.0 and after an initial drop in rankings (4-5 places on top keywords from rank 1), I have seen a small climb in rankings by a couple places but have seen a huge jump in my DA, PA and Domain trust/rank... That FAR surpasses any other magicians website. However I am ranking below some really really poorly rated sites. I am getting a PA of 51 which is really high for the industry (maybe the highest). If this is the case, why am I not number one for every keyword in my demographic?
Moz Pro | | TomLondon
I know this is asked a lot but how reliable are Moz stats, I know no one can really know how Google rank sites on SERPS but if Moz is the best shouldn't it be kinda close? Any thoughts on how accurate Moz rank is and also if it can be used as some indication of the SEO power behind a website.0 -
Is there any way to get historical API data?
I'm looking for a way of requesting domain authority and some other stats via the API for several previous months. I noticed on here: http://apiwiki.seomoz.org/what-is-the-mozscape-api
Moz Pro | | ReallyEasyCart
It says "SEOmoz PRO members now have access to historical API linking data - read about this great new feature on our blog!" but i could not find this blog post anywhere and maybe I'm misreading this line anyway.0 -
Difference between SEOmoz Pro and Mozscape API
Hi, I probably put this question in a wrong category, so I'd like to ask again. I am new in SEOmoz and writing this from Japan.
Moz Pro | | flaminGoGo
I have couple of questions and really appriciate if you can help me to understand SEOmoz. better. First of all, what is the difference between SEOmoz Pro and Mozscape API? I signed up for 30days free trial and now continue being $99 PROmember.
Is that correct that rate limit is one request every 2 seconds with PRO? and also want to know about free API that I found after signed up for PRO.
Is there any restrictions on free API except for rate limit is one request every 10 seconds? The main reason I signed up for SEOmoz was to use the API link metrics, and because of the language I got really confused now.0 -
Using api to find likes, tweets and shares
Hi all, I have a question regarding using the API to get data on a list of urls and not one by one. I am not a developer so I might not grasp the API so well. Basically I have a list of urls, what I need from each is the following information: <colgroup><col width="98"><col width="89"><col width="92"><col width="111"><col width="99"><col width="50"><col width="64"></colgroup>
Moz Pro | | rightmove
| Page authority | linking root domains | total links | Facebook Shares | Facebook likes | tweets | Google+1's | I found a google doc that gives me SOME of the data but not all (also it might be old as it uses the linkscape call (I dont know if thats changed)) Any idea how I can tune my googledoc spreadsheet to include the additional metrics? Thanks 🙂0 -
Mozscape API call returning no json
Hi - I'm having issues getting started with the Mozscape API. I'm putting the following into my browser: http://[member-ID]:[secret key]@lsapi.seomoz.com/linkscape/url-metrics/greatbritishchefs.com According to the blog posts I'm reading I should get loads of json script returned but all I'm getting is {} Does anybody have any pointers as to what I might be doing wrong? Thanks
Moz Pro | | GBC0