Mozscape Index update frequency problems?
-
I'm new to Moz, only a member for a couple months now. But I already rely heavily on the mozscape index data for link building, as I'm sure many people do. I've been waiting for the latest update (due today after delay), but am not seeing any mention of the data yet - does it normally get added later in the day?
I'm not that impatient that I can't wait until later today or tomorrow for this index update, but what I am curious about is whether Moz is struggling to keep up, and if updates will continue to get more and more rare? For example, in 2013 I count 28 index updates. In 2014 that number dropped to 14 updates (50% drop). In 2015, there was only 8 (another 43% drop), and so far this year (until the March 2nd update is posted) there has only been 1.
This isn't just a complaint about updates, I'm hoping to get input from some of the more experienced Moz customers to better understand (with the exception of the catastrophic drive failure) the challenges that Moz is facing and what the future may hold for update frequency.
-
I wish I had an answer to that
If I had to guess, I'd say sometime in the next 2 years, but not anytime in the next 9 months. We've stopped asking the team for a final delivery date because it's just too hard to estimate all the work required, and past estimations have been so far off target. Instead, we just try to estimate the next quarter worth of sprints and then measure how we perform against those.
It turns out, replicating a processing system like Google runs without billions in revenue is really hard
-
Hi Rand,
I appreciate the official response, and I enjoyed reviewing the new data shortly after my post.
It's especially great to hear about the dedicated team working on the next generation of analytics gathering, I think that will go a long way to silencing critics of the current index processing system (like me). I can't wait, are you able to give us any kind of timeline for that migration?
-
Hi Kevin - the index update should be live as of right now (probably only a few hours after you posted this message). We aim to have one index update per month, so 12 per year. We had a catastrophic failure on our early January index, so it had to be abandoned (noted here: https://moz.com/products/api/updates), but the team has been working hard to fix issues and prevent others from arising. Unfortunately, it's often the case that we encounter new/unexpected/never-before-seen issues that need to be addressed. Frustrating, but unavoidable as best we can tell. Obviously, we will continue to do our best to get these indices out on time.
As far as the future goes, it's hard to say. We have a bigger team now than we last year -- 4 folks work full time on the Mozscape index and 4 are working on the next-generation version of the index (which will update in near-real-time), and we certainly have much better monitoring and operational structures in place. But, as I noted above, it seems that the errors/issues we face are always new and unique - things we've never seen before in the 100+ index runs we've had over the last 8 years. I can tell you we're building processes to identify problems before they happen, and that we're better staffed, and that we have engineers on-call 24/7 to fix issues if they crop up, but processing full-graph metrics on a full-scale web index whose shape and composition can vary wildly means there's still uncertainty and probably always will be. Our job is to keep reducing that uncertainty and finding optimizations, while we rebuild the full system in the background to eventually replace the old, batch-processing system that's at the core of so many of our challenges.
Hope that helps.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Problem not showing stars
من از اسکیما استفاده می کنم. اما در سایت من نمایش داده نمی شود. به عنوان مثال " خرید ساعت " البته برخی از مطالب در سایت اصلی من ستاره دار شده اند. قابل مشاهده در آدرس: دیجی ساعت
API | | aliyousefiisfsfs0 -
How frequently is the Search Volume update for each keyword? API for Search Volume?
Subject pretty much says it all... How frequently is the Search Volume update for a given keyword? Is there an API call that would include keyword-specific Search Volume for one or more keywords? Thank you.
API | | ToddLevy0 -
Why the Feb 2018 update was so early?
Hi There! We are using Moz to compare our metrics to increase our SEO / SERP penetration. According to MOZ API Updates, it was mentioned that the next update will be on 26th Feb. But the update was early, could you please let us know the reason for the same. Why is there a discrepancy between the date mentioned for the Moz Update and the date of release? Thanks Malik Zakaria
API | | mzakaria0 -
Why does OSE show old data (Previous update results)?
Moz api started to show July 13 update results for my website. I checked it 2 days ago and saw all new established links and updated DA PA for July 27 update. But last 2 days both Moz Api and OSE main page show July 13 update results. Is there a maintenance or mismatching error between old and new databases?
API | | cozmic0 -
How to retrieve keyword difficulty information using Mozscape API?
Hi, Are we possible to use Mozscape API to retrieve keyword difficulty information for a list of keywords? I can't find its documentation. Thanks
API | | uceo0 -
March 2nd Mozscape Index Update is Live!
We are excited to announce that our March 2<sup>nd</sup> Index Update is complete and it is looking great! We grew the number of subdomains and root domains indexed, and our correlations are looking solid across the board. Run, don’t walk, to your nearest computer and check out the sweet new data! Here is a look at the finer details: 141,626,596,068 (141 billion) URLs 1,685,594,701 (1 billion) subdomains 193,444,117 (193 million) root domains 1,124,641,982,250 (1.1 Trillion) links Followed vs nofollowed links 3.09% of all links found were nofollowed 62.41% of nofollowed links are internal 37.59% are external Rel canonical: 27.46% of all pages employ the rel=canonical tag The average page has 92 links on it 74 internal links on average 18 external links on average Thanks again! PS - For any questions about DA/PA fluctuations (or non-fluctuations) check out this Q&A thread from Rand:https://moz.com/community/q/da-pa-fluctuations-how-to-interpret-apply-understand-these-ml-based-scores
API | | IanWatson7 -
Mozscape Index
Hello: There was a Mozscape Index scheduled 9/8/2015 and now it go pushed back October 8,2015. There seems to be a lot of delays with the Mozscape Index. Is this something we should expect? Updates every 2 months instead of every month? Thanks!
API | | sderuyter1 -
On Page Grader Problem-Sorry But This Page Inaccessible
Greetings: When I try to use the on page grader and enter my URL, an error message appears stating: "Sorry But This Page Inaccessible". The URL is http://www.nyc-officespace-leader.com/commercial-space/office-space and it works fine when I enter it on my browser. Any page from this domain generates this error. Is there a bug with this tool? How would I go about tracking ranking on various keywords? I see it is possible to tag keywords, and I have done so for about 250. But I don't know how to generate a ranking report for these keywords; ideally I would like to do so filtering them by the label I have applied. Any suggestions? Thanks,
API | | Kingalan1
Alan0