Does OSE crawl our site often? Or do we have an other problem? (internal links)
-
I am wondering if i can find out when OpenSiteExplorer crawled our entire website for the last time.
A few months ago we added some internal textlinks to all of our productpages. Those are links to the brandpage of that particular product. Some of our brands do have up to 1500 products, so there should be at least 1500 internal links pointing to that brandpage.But OSE still gives me a wrong count. It says there is only 1 internal link pointing to that brandpage.
Is it because OSE never crawled our site again? Or am i missing something here?(maybe too many internal links to one specific brandpage is not okay?)I'll hope you guys can help me out with this one.
-
Hi there!
For our inbound links, here is how we compile our index:
<code>- We grab the most recent index. - We take the top 10 billion URLs with the highest MozRank (with a fixed limit on some of the larger domains). - We start crawling from the top down until we've crawled 90,000,000,000 pages (which is about 35% the amount in Google's index).</code>
Therefore, if the site is not linked to by one of these seed URLs (or one of the URLs linked to by them in the next update) then it won't show up in our index. Sorry!
We update our Mozscape Index every 4 weeks. Crawling the entire Internet to look for links takes 2-3 weeks, but our crawlers are always collecting data. When we need to put the index together, we grab all the data they have collected and start processing which can take up to 3 weeks to determine which of those links are the most important. You can see our most recently updated schedule here: http://moz.com/products/api/updates
Mozscape focuses on a breadth-first approach. Therefore we almost always have content from the homepage of websites, externally linked-to pages, and pages higher up in a site's information hierarchy. However, deep pages that are buried beneath many layers of navigation are sometimes missed and it may be several index updates before we catch all of these.
If our crawlers or data sources are blocked from reaching those URLs, they may not be included in our index (though links that point to those pages will still be available). Finally, the URLs seen by Mozscape must be linked-to by other documents on the web or our index will not include them.
For now, the best thing you can do to help your domain become indexed is to work on link building for links from sites with high mozrank.
Hope this helps clear things up!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to download full list of internal links properly with OSE?
Hi Moz Community, I am having issues getting a full list of my internal links when I go to the Open Site Explorer. When I export the internal links into excel, I am getting half of my URLs and 80% of what I am getting is absolutely crazy URLs that are super super old. Plus, these old URLs are not even showing in my campaign crawls. So the crawls themselves are different. Any help on this would be greatly appreciated, since I am trying to get a full picture on the internal link structure of my site. Thanks!
Link Explorer | | RyanRhodesGW0 -
Can I check outbound links from a website?
Hi! Is there an easy way to check if a certain domain has a link to my domain. Is there something similar to "checking outbound links" on a website?
Link Explorer | | AnnabelH0 -
Open Site Explore page titles show as "No Title"
This has been asked many times but I cannot find an answer. On Open Site Explorer, most URLs I enter show as "No title" which means you have to hover over the URL to see which page is being referred to. I know you'd want an example, so here's your own site Moz.com 🙂 CWrkxZQ
Link Explorer | | clifra0 -
No backlinks reporting in MOZ OSE?
We have a bunch of links and none of them report in MOZ OSE. Does the MOZ OSE tool have a cache that takes a while to refresh? We have recently moved over to CloudFlare and since the move the numbers do not add up. Here is a link to a ahrefs screenshot. Any insights would be greatly appreciated. moz.png
Link Explorer | | seanallen0070 -
Only one internal Equity Passing Link
Our web site IYBI is reporting only one internal equity passing link. I've somehow never noticed this and now that we are doing a lot more competitor analysis, I'm a little concerned given the numbers some of the other sites in the space are getting. I'm not sure I understand it completely and how it's possible we only have one. Any help would be appreciated.
Link Explorer | | wearehappymedia0 -
Does Feedburner URL of the Home Page Carry Link Equity?
Hi There, During an SEO Audit, I found that OSE categorizes Feedburner URL of root domains under link-equity passing and followed. For example, the following link has been categorized under link-equity passing and followed: http://feeds.feedburner.com/SpoonflowerBlog I have heard that a lot of SEOs saying feedburner links don't carry any link juice. If that's true, then why does OSE categorize feedburner URL of root domains under link-equity passing and followed? I would appreciate if someone from the Moz staff could take some to answer this. Thanks.
Link Explorer | | TopLeagueTechnologies1 -
Sidebar on OSE causing problems
For the last few days a sidebar has been showing up on OSE and as a result I cannot see the social information on URLs, can someone fix this?
Link Explorer | | Eavesy0 -
External Followed Links report
I would like to know, how do I get a report with the TOTAL amount of my external followed links.When I look on "Link Metric" it shows me: External Followed Links 2,951
Link Explorer | | AutoEuropeBut when I looked on Inbound links (only follow / only external), it shows me only 645. Why is that and how do I get the total from 2.951?
Thanks
0