Does OSE crawl our site often? Or do we have an other problem? (internal links)
-
I am wondering if i can find out when OpenSiteExplorer crawled our entire website for the last time.
A few months ago we added some internal textlinks to all of our productpages. Those are links to the brandpage of that particular product. Some of our brands do have up to 1500 products, so there should be at least 1500 internal links pointing to that brandpage.But OSE still gives me a wrong count. It says there is only 1 internal link pointing to that brandpage.
Is it because OSE never crawled our site again? Or am i missing something here?(maybe too many internal links to one specific brandpage is not okay?)I'll hope you guys can help me out with this one.
-
Hi there!
For our inbound links, here is how we compile our index:
<code>- We grab the most recent index. - We take the top 10 billion URLs with the highest MozRank (with a fixed limit on some of the larger domains). - We start crawling from the top down until we've crawled 90,000,000,000 pages (which is about 35% the amount in Google's index).</code>
Therefore, if the site is not linked to by one of these seed URLs (or one of the URLs linked to by them in the next update) then it won't show up in our index. Sorry!
We update our Mozscape Index every 4 weeks. Crawling the entire Internet to look for links takes 2-3 weeks, but our crawlers are always collecting data. When we need to put the index together, we grab all the data they have collected and start processing which can take up to 3 weeks to determine which of those links are the most important. You can see our most recently updated schedule here: http://moz.com/products/api/updates
Mozscape focuses on a breadth-first approach. Therefore we almost always have content from the homepage of websites, externally linked-to pages, and pages higher up in a site's information hierarchy. However, deep pages that are buried beneath many layers of navigation are sometimes missed and it may be several index updates before we catch all of these.
If our crawlers or data sources are blocked from reaching those URLs, they may not be included in our index (though links that point to those pages will still be available). Finally, the URLs seen by Mozscape must be linked-to by other documents on the web or our index will not include them.
For now, the best thing you can do to help your domain become indexed is to work on link building for links from sites with high mozrank.
Hope this helps clear things up!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Question about link report
Hey all, SEO noob here, back with another question. I downloaded a link report, and I don't understand the columns Links to Page, Outbound Domains from Page, and Outbound Links from Page. Is Links to Page the number of links from the source page to the target page? Is Outbound Domains from Page the number of domains linked to by the source page? Is Outbound Links from Page the number of outbound links to any domain or page from the source page? Thanks in advance! AK
Link Explorer | | AndyKubrin0 -
Are Unlinked Mentions Coming Back in the New OSE?
Hi Mozzers, Just a quickie, with the unlinked mentions being retired, I was wondering if Moz have any plans to bring this feature back through the new link explorer? Many thanks! 🙂
Link Explorer | | Corbec8881 -
Is open site explorer the best way to find backlinks?
I know my websites have links from other sites, but I don't see it listed using the open explorer tool. perhaps the reputation of the external link is a factor? _Cindy
Link Explorer | | cceebar0 -
Help needed interpreting inbound link analysis please
Hi I am trying to determine how effective our store blog is and am trying to analyse the inbound links it provides to our store domain but am confused with the results, can anyone help please? For information the blog is on a subdomain of the store site, has plenty of relevant original content, and has been regularly updated over several years in line with basic Yoast SEO best practice. When I look at analysis of inbound links and linking domains for the store site in Moz Pro, the blog subdomain does not show as providing any inbound links. Google Search Console shows similar results, with no inbound links to the store domain showing from the blog subdomain in either 'search traffic > links to your site', or 'search traffic > inbound links'. Open Site Explorer for the store domain also shows no inbound links from the blog subdomain to the store with 'link source > only external' selected. But if I change to 'link source > only internal' then all of the inbound links from the blog subdomain to the main store that I would expect to see appear. Should I be concerned by the fact I can only find these inbound links under this one specific search and not in any of the others? Does this suggest a problem in the passing of any current and future SEO benefit from subdomain blog to main domain? Or should I not worry about it too much? I am looking to begin a full overhaul of all SEO for the store and blog from the ground up and want to be sure the connections between the blog subdomain and store are functioning properly before I start. Any help in unpicking this would be really appreciated Thanks very much
Link Explorer | | Warren_330 -
Is OSE having problems?
The healthpage says it is operational but I'm getting a ton of 'There was an error getting your data' messages. First I thought, it just happens to smaller pages, since testing it with moz.com worked. But even pages like nbc.com give me this error. I tried logging out an in again and even swithcing browsers - without success. Anyone else having problems like this right now? Cheers, Chris
Link Explorer | | chrwald0 -
OSE Getting Less Accurate
It seems that the more links I get the lower my DA is. My DA has been dropping for some time now as I gain more backlinks. It also does not seem to pick up anywhere close to the number of domains that is linking to my site. My url is hiddentriforce.com GWT says I have over 600 domains linking in. The majority of the links are from news or articles that we cover which other sites pick up and link to us as the source. We also have a ton of scholarship links from .edu and education sites from a giveaway that we do every year for students. Why does OSE seem to lower my DA the more I get off-page factors to the site?
Link Explorer | | Atomicx0 -
My facebook stats are not showing up in open site explorer when I enter my website
My facebook stats are not showing up in open site explorer when I enter my website (www.transformationstreatment.com). I have over 15k likes on my facebook page but the open site explorer shows zero. If someone could please help me with this I would greatly appreciate it.
Link Explorer | | vrluis9k000 -
Which links are most powerful
If I am analyzing a competitor website, how can I see which websites are the most powerful to get links from? I am using Opensite Explorer.
Link Explorer | | vkpolar0