Why does Linkscap API request hang while extracting data ?
-
Hi,
I am using LinkScape API to get follow and nofollow links .
I use cron to get data for each url of sitemap.xml.
However while cron is running, the extraction of data hangs on some pages which i later need to delete manually for re starting the execution.
Do anyone have any idea why this is happening ? How can i ignore such pages ?
-
Hi...
Thank you so much for your reply..
I was implementing the changeswhich may work.. but they didn't.
I recently came to know about updates to seomoz api which are like,
- SEOmoz Free API: 1 request for every 10 seconds
- Free Extended and PRO Members: 1 request every 5 seconds
- SEOmoz Site Intelligence API: 1 request every 2 seconds
I think this will help me a lot .. thank you ..
-
Hey Ravi,
Sorry for the delayed response - I wanted to follow up with the engineers to see if they had any suggestions for you.
They agreed the Limit parameter set to 1,000 might be too large to process. Have you tried adjusting that to 300 or even 500? Do you see better success at a lower limit?
Our system will timeout at about 60 seconds so I'm not sure if the hanging is on our end. If dropping the limit size doesn't help, you might want to think about ending the request after about a minute. Sometimes requests that are too long wil timeout, but work fine on a retry as some data will be cached from the previous request.
I hope this information is helpful, but let me know if you're still experiencing issues!
Thanks,
Carin -
Some pages are the pages which are moved or not found. I am trying to ignore such pages by getting http response code. But this process takes large amount of time. Almost 10 minutes for a single url of sitemap.xml.
The url of call is like : (php)
http://lsapi.seomoz.com/linkscape/links/URL HERE?SourceCols=4&TargetCols=4&Scope=page_to_page&Sort=page_authority&Limit=1000&Filter=follow&AccessID=" . $accessID . "&Expires=" . $expires . "&Signature=" . $urlSafeSignature;
Thanks .
-
Hey! This is an issue I haven't heard of before - would you be able to provide anymore information like an example query to the API and some of the pages you are seeing hang?
Thanks!
Carin
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Keyword Rank History chart has missing data for periods of time
Hello, When I look at the rank history for a particular keyword, there are missing gaps in the data. For example, the keyword was fluctuating around rank 10-20 from Jan 2022 to July 2022. Then suddenly the line on the graph disappears. It starts tracking again in Oct 2022 starting at rank 16. It almost appears as if Moz lost track of the keyword for some time. Other tracked keywords did not experience the same issue during this time. Then as a result, Moz reporting will say the tracked keyword moved up in ranking 40+. What is happening here? I'm unsure how to explain this activity to my clients.
Moz Pro | | celsiusmarketing0 -
No data in Moz
Hello, Since last tuesday there has been no data in Moz anymore. The error code for the homepage says: '608 : Page not decodable as specified Content-Encoding:' I checked this list http://moz.com/help/pro/why-can-t-rogerbot-crawl-my-site but it doesn't tell me more.
Moz Pro | | MarcelMoz
Hopefully anyone can help me. Thanks, Marcel0 -
Moz vs google data conflict?
Hi there, I am doing an SEO site audit for a client(giveaway, and here is the problem: when performing site:domain.com on google --> 13,800 pages were found When I see this number it seems to be a bit too much compare to the links i checked on integrity(link check for broken links) which gave me a result of 1291. I digged in more into the Google results and saw hundreds(maybe thousands) of pages that are blocked by robots.txt. So I am thinking, ok this is it, thousands of pages can't be crawled by the search engines. Here is the big BUT though, then I check at my moz crawl (see attachment) and no pages are blocked by the SEs, and then look at the dups, only 23 recorded?? Is Moz not crawling properly the 13,800 results that google finds or is this some magical phenomenon happening here? I am really confused here that is why I need some help here! Thank you guys! A990Hu4.png k842AOn.png
Moz Pro | | Ideas-Money-Art0 -
Linkscape update 2013
After the last linkscape update of 2012 (December) I noticed that it had missed a couple of high quality follow links to other sites that I'd built. I just assumed that the crawler had missed them but would pick them up next time. It hasn't though (I've been built some since then as well) and my DA has stayed the same. Help...
Moz Pro | | EmpofMan0 -
"Does not respond to web requests" error
When trying to set up a new campaign I get the following message:
Moz Pro | | bshanahan
"Roger has detected a problem: We have detected that the domain www.chicagofinancialadvisers.com does not respond to web requests. Using this domain, we will be unable to crawl your site or present accurate SERP information." Can someone please tell me what I need to do on my site to make this work? I haven't seen this before and have done many other campaigns. Thanks a lot!0 -
API Request Rate Problem
Hi there A Java app of mine worked perfectly until recently, when it gets "rate exceeds your current plan" errors from the API, although it does only 1 request in 10 seconds. Any idea what's wrong? Cheers, Chris
Moz Pro | | Diderino0 -
Why is my traffic data not showing any scores at all?
Ive recently joined seomoz and i am receiving data regarding my keywords but the traffic data is just flat lined, no information at all. I know am getting hundreds of hits a day through my own web analytic and webmaster tools but nothing from seomoz. any help please?
Moz Pro | | totaldriveways0 -
Opensiteexplorer hangs at 1051 links...
...and won't complete the download. I have been extremely patient, but it's stuck there no matter how many times I try or long I wait. Ideas?
Moz Pro | | rtora0