Can we add sites to the crawl queue for OSE?
-
Is it possible to request that Open Site Explorer crawls a new URL on its next run?
This tool is the first place I go to when working on a new site, and when there is "No Data Available" this is a little frustrating.
I fully appreciate that this lack of data is usually a signal that the website is either very new or of low quality, however that if often the reason that I am brought in and would very much like to benchmark and provide initial analysis using this tool.
It would make sense that OSE crawls the sites that Moz members are working on wouldnt it?
Scott.
-
Hi Scott,
Derek and Brad are correct in that there's no real way to tell Roger to index a specific page.
Your best bet to get your page index in OSE is to work at building quality backlinks and making sure your site is configured properly so we can reach your pages.
Also remember you can set up your campaign for your site and crawl it that way if you want to evaluate the rest of your page information.
I hope that helps.
Cheers,
Joel. -
You "ask it what it knows" when you type in a URL and hit Search. Think of it like walking up to a really smart person and asking them to list off every book every written that includes the word "rhinoceros." They can only list the books they've read. A person only has so much time for reading, and OSE only has so many resources for crawling the web. It will never get everything.
If OSE doesn't show any backlinks for a URL, it means it didn't see any backlinks in the portion of the web that it crawled. That might mean there are no backlinks at all, or it might mean the backlinks are yet undiscovered (probably because they're in an obscure corner of the web). The best suggestion is to use multiple tools, as Derek says below.
-
SeoMoz members have no control over what Mozbot crawls. Your best bet is to use a variety of site explorers to find links. Ex. Majestic, Google Webmaster tools, Link:www.example.com, Bing Link Explorer
-
Yes I know what you mean here, but if there is any way to "point" the crawl engine at the site and ask it what it knows?
I hadnt thought about it from that perspective, just asking the question to see what is possible to try and get more data if it can be got?!
-
As far as I know, crawling your website wouldn't accomplish anything. OSE would need to crawl the pages that link to your website. And if you knew what those were, you wouldn't need OSE, right?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
SEO Crawl Report Images?
Does SEOMOZ crawl images in the report? Raven tools is showing me about 200 missing alt tags and title tags. I can not seem to find any of this information on the SEOMOZ report. Am I missing something?
Moz Pro | | jasonsixtwo0 -
Only One page crawled..Need help
I have run a website in Seomoz which have many URLs with it. But when I saw the seomoz report that showing Pages Crawled: 1. Why this is happen my campaign limit is OK Tell me what to do for all page crawling in seomoz report. wV6fMWx
Moz Pro | | lucidsoftech0 -
Why is my crawl STILL in progress?
I'm a bit new here, but we've had a few crawls done already. They are always finished by Wednesday night. Our website is not large (by any means), but the crawl still says it's in progress now 3 days later. What's the deal here?!?
Moz Pro | | Kibin0 -
Crawl Diagnostics Error Spike
With the last crawl update to one of my sites there was a huge spike in errors reported. The errors jumped by 16,659 -- majority of which are under the duplicate title and duplicate content category. When I look at the specific issues it seems that the crawler is crawling a ton of blank pages on the sites blog through pagination. The odd thing is that the site has not been updated in a while and prior to this crawl on Jun 4th there were no reports of these blank pages. Is this something that can be an error on the crawler side of things? Any suggestions on next steps would be greatly appreciated. I'm adding an image of the error spike Xovep.jpg?1 Xovep.jpg?1
Moz Pro | | VanadiumInteractive1 -
Total number of links in OSE not clear to me.
I have compared 3 sites in open site explorer. The total # of links in the subdomain metrics report for one of the urls is 46,787. Where is that number coming from? There's 125 total external followed links and 132 total external links. I cannot see the number for internal links, but I'm sure it isn't 40000+. So 46,787 is the result of the addition of what? Thanks a lot for your help, and sorry for this newbie question 🙂
Moz Pro | | gerardoH0 -
Any SEO moz users notice a HUGE change in OSE (Open Site Explorer) link data numbers?
Hi All, I am having some serious concern with OSE data recently for numerous clients, one client I want to talk about today has the following data from OSE for the month of August 2011 compared with July 2011: Total links to the domain: (decrease of around 100,000+)
Moz Pro | | ColumbusAustralia
External Followed links: (decrease by around 5,000)
**Linking Root domains: (decrease of over 60) ** The crazy thing is that the domain authority has actually gone up by around 5 points for this client even though every thing has suddenly gone down? Also funny thing is we have been link building quite strong for this client over the last 12 months using only high quality sources from out niche. I am worried that their is serious issues with the data, I realise we saw some updates to OSE recently yet I am suprised it can be this drastic. Kind Regards. PSV1 -
Page Rank and offline sites
I have a domain with PR6 according to the Historical Pagerank Checker. But that last PR was calculated 2 years ago. I brought the site back online a few days ago and have checked that many/most of the backlinks are still valid. It is now in the Google index but the Historical Pagerank Checker shows PR0. Will it get back its previous rank or something close to it? How long will it take?
Moz Pro | | DomainOptions0 -
Can anyone edit their comments?
I've been unable to edit my comments here for about a week 😕 Just me or is it like this for everyone? Tried FF 4, Chrome, IE 9, Safari and iPhone Safari. Just get the loading icon. If it's just me I'll submit a bug report.
Moz Pro | | StalkerB0