Thoughts on scraping SERPs and APIs
-
I'm perplexed about a couple of things.
Google says that scraping keyword rankings is against their policy from what I've read. Bummer. We comprise a lot of reports and manual finding and entry was a pain. Enter Moz! We still manually check and compare, but it's nice having that tool. I'm confused now though about practices and getting SERPs in an automated way. Here are my questions
- Is it against policy to get SERPs from an automated method? If that is the case, isn't Moz breaking this policy with it's awesome keyword tracker?
- If it's not, and we wanted to grab that kind of data, how would we do it? Right now, Moz's API doesn't offer this data. I thought Raven Tools at one point offered this, but they don't now from what I've read. Are there any APIs out there that we can grab this data and do what we want with it? (let's day build our own dashboard)?
Thanks for any clarification and input!
-
You're much too kind Happy New Year!
-
Hi Rand. Anyone ever tell you you're awesome?
I just did. Thanks bud
-
Hi Boogily,
A) In terms of ToS, I'm not 100% clear. I know that Google has been strict about automated rank checking programs installed on a user's local machine, and they've enforced that a bit through technical and guidelines means. However, they've been unclear about companies like Moz, Searchmetrics, GetStat, AuthorityLabs, Conductor, Brightedge, Yield, RankAbove, and hundreds of others which provide this data in their software. I suspect they don't love it, but tolerate it and haven't taken action for over a decade now.
B) On the API front, check with GetStat and AuthorityLabs. I believe they both have rank data offerings from an API standpoint. Moz doesn't offer that and probably won't in the future, either.
Cheers!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Thoughts on the Keywords sending visits.
So, many of you know my current bizarre situation. We are getting some assistance. If you are curious my posts are on here and you have all been extremely helpful. To my question: So in my Moz campaign I am tracking a handful of keywords and such, and then there are all the keywords that are sending results currently. Some of these I feel are pretty reasonable to transfer over to my keyword list. Some are pretty obvious that we should not use. Then there are some that seem to be luke warm. Are then worth tracking just because they are sending traffic? I guess the better question would be to look at if they are actually converting. Are people who are looking for Ice Cream upset when then end up at a place that doesn't sell Ice Cream.
Moz Pro | | HashtagHustler0 -
Is there a way to get moz keywords analysis % through the api ?
I'm looking to integrate moz keywords analysis grade (in %) into my keyword research spreadshit as a column. Is there a way to get the % through the API ? Thanks
Moz Pro | | Sindicic_Alexis0 -
Paid API Access
We've been using your API on regular basis over the past few months, when lately we started having problems accessing the data it provides.I received an empty object when tried fetching info using the following URL:string sUrlToScan = "http://lsapi.seomoz.com/linkscape/url-metrics/"+sHost+"?AccessID=member-d73bd20330&Expires=1355756373&Signature=....";* Please note - I removed the Signature field for security reasonsI would really appreciate it if you could let me know why the service has stopped working.Is it because the 'Expires' is no longer valid?Could you please provide me with the up-to-date AccessID + Expires + Signature values so I could continue using your API?Thanks!
Moz Pro | | Sorezki0 -
How to get the total external links to a page and total external links to the domain using Mozscape API? I could not see an option in the bit flags
Hi, I was trying to get the data for total external links to a page and total external links to the domain using Mozscape API but I can't see a bit flag which can do that. There are bit flags for external followed links to a page and external followed links to a domain but I wanted the total external links data, is there a way to do that using Mozscape API else I would end up copying the data manually from OSE which would be cumbersome and time consuming. Your help is highly appreciated.
Moz Pro | | HQP0 -
SEOmoz ranking report SERP
I was wondering how SEOmoz the SERPs tracks? e.g. the SERP of keyword in the Google US report, doesn't give me the same result as https://www.google.com/search?pws=0&gl=us&q=keyword Does SEOmoz check the google.com SERPs from several locations and calculate an average?
Moz Pro | | Teklan0 -
API callback parameter
I'm using the free API, and would like to use jQuery to fetch the JSON data outputted by the API. I need to make a JSON-P request, and for this I need to be able to pass some form of callback parameter in the url. I'd like to know if this feature is available, and if so what parameter I need to specify in my request. Of course I could just CURL to the API, but that's really a fallback for if I can't get the data using JS. Thanks
Moz Pro | | jgt19850 -
Campaign data via API
I am currently in the process of building an in house system for managing our clients SEO campaigns. One of the features that we will be including is a customer login where clients can view their reports and site progress. We plan on providing rankings and traffic stats within this which we will collect on a weekly basis to store historical changes. While I could integrate into analytics for traffic and create my own web crawler for rankings, this seems a waste since I already have that data in one central location. Looking through the moz API details it seems to have lots of functions for links but nothing relating to the campaigns. Is there any way or are there any plans to open up campaign data to an API? If I could do this I would certainly upgrade to allow each client to have their own campaign.
Moz Pro | | PPCnSEO0 -
Which Google API do you use to creat a rank checking tool?
I can't seem to work this one out. Basically I want to make a very simple rank checker tool something similar to http://www.mikes-marketing-tools.com/ranking-reports/ where a user simply type in a domain and a keyword and in return is shown the position within the results. The problem is I can’t work out which API's I need in order to build it, does anyone know what I need to poke about with? It must work with Google and ideally with Bing and the rest
Moz Pro | | therealmarkhall0