How Is a Page Crawled by Moz When Moz Says 'No Links'?
-
As above, really. I've crawled a new client's site to find the Moz crawler has identified a handful of 404 errors.
The Moz crawler says these pages have '0 linking domains', and OSE has no data for these pages.
So how are these pages being crawled by Moz and what should I advise my client?
-
Thanks again, Peter.
-
That's great, thanks Keri.
-
Hi, no problem. Keri has answered your query re the Moz crawler not finding them.
Peter
-
If you download the CSV report, we list the referring page there. We know it isn't obvious and we're working on making this a little easier to find.
-
Hi Keri,
Thanks for getting back to me (thanks to Peter above, also).
So what is the best way of me finding where these 404 pages are linked from?
I am 99% sure they are internal links that I will ask the client to remove, but would like to avoid manually checking every link on the site if possible.
-
Hi Peter,
I searched for their domain name and about the expected number of results came up.
I searched for these individual URLs that Moz is reporting using the site: command, and they were not returned.
They are also neither in the site's sitemap (HTML page or XML file).
So how is SEOmoz Crawler finding them?
-
We have two completely different crawling systems. We have on crawler for your website, where you enter in your URL and we crawl your site looking for problems. We have a different crawler for OSE that goes and looks at the web in general and crawls for links. The two crawlers do not talk to each other, so it is actually understandable that OSE may not have data on the site. OSE doesn't have the same index size as Google, so we don't see all the links that are actually out there (though we're working all the time on improving this!).
-
Hi, if you put site:xxx.com into Google where xxx.com is your client's domain name, does Google report anything?
Peter
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Moz tool to estimate a website's organic traffic?
Moz has tons of great tools, so I was quite suprised there is nothing similar to Alexa's siteinfo tool, or SimilarWeb's site comparison tool. Am I missing something? Or does such a tool exists at Moz? Or is there a different way of estimating my competitor's traffic? It would be great if Moz would have something like that.
Link Explorer | | GrandRapids0 -
Can I check outbound links from a website?
Hi! Is there an easy way to check if a certain domain has a link to my domain. Is there something similar to "checking outbound links" on a website?
Link Explorer | | AnnabelH0 -
Moz Spam Score 9/17 when there are no links
Hi I was just looking at a competitors new site in OSE and it had a DA and PA of 1 which I would expect and also no links, which again I would expect, basically the guy used to design stuff for me and thinks he can do a better job so he is giving it ago. But he has a spam score of 9/17 but no links - how is this even possible. Thanks Andy
Link Explorer | | Andy-Halliday0 -
Why Aren't All My Backlinks Appearing in Open Site Explorer?
Cross-referenced on Google Webmaster Tools and it looks like MOZ isn't pulling all my links. Any reason why? Thanks in advance!
Link Explorer | | Dental-Care-Allilance0 -
Just Discovered Links Are Down for ~2-3 Days
UPDATE 12/01/15: This issue is now resolved and Just Discovered Link data is back. Sadly, I've got bad news. Due to an error we made (literally a typo on a curl command), Just Discovered Links in Open Site Explorer will be unavailable for 2-3 days. The data was actually lost, but we're re-indexing all those links now and they should be back in working order by Monday. NOTE: this only affects link counts and links listed in the "Just Discovered" sections of OSE. No links in the main index nor any metrics (PA/DA/etc) are affected. Our sincere apologies - we're going to build mechanisms to prevent this for the future.
Link Explorer | | randfish3 -
Learn how to use Open Site Explorer's Top Pages report to help inform your content marketing efforts. Get your Daily SEO Fix!
With the Top Pages report, you can see the pages on your site (and your competitors’) that are top performers. The pages are sorted by Page Authority - a prediction of how well a specific page will rank in search engines - and also metrics for linking root domains, inbound links, HTTP status and social shares. Be sure to watch today's Daily SEO Fix video tutorial to learn how to use Open Site Explorer's Top Pages report to analyze the competitions' content marketing efforts and to inform your own. This video is part of The Moz Daily SEO Fix tutorial series--Moz tool tips and tricks in under 2 minutes. To watch all of our videos so far, and to subscribe to future ones, make sure to visit the Daily SEO Fix channel on YouTube.
Link Explorer | | kellyjcoop3 -
Why Moz Back links check tool is very Slow compared with other tools?
I've created a new website 50 days old with a vision of sharing high-quality content. I created some pages which are useful and helpful, what make other people link to my site. Early days, two weeks ago I've started checking my backlinks starting with Ahref and majestic. three days ago I decided to join Moz too and use Moz explorer. But, I discovered that Moz tool is too slow compared with other tools. Why? Starting with Ahref. They discovered more than 100 links and updating their data for my site as a new website every 6 hours as I know. Also majestic for the first time discovered six links then every two days they updating their data. Even Google discovered some of my backlinks. Specially links from high-quality sites such as Wikipedia with high-quality pages. Moz tool discovered zero of my backlinks even after subscribing. I don't think Moz need to discover websites like Wikipedia which already indexed by Moz. So, I wonder Why Moz tools are very very very slow like that? Update: (This is a question, not a discussion. So, please don't edit it) Can any Moz support answer please? Thanks.
Link Explorer | | Eslam-yosef0 -
Page Authority Migration
PageA.html has a lot of links and has a Page Authority score. It is 301 redirected to PageB.html and eventually in OSE this also accumulates a Page Authority score. Is there a reason why PageA.html many months/years after continues to have a Page Authority when it is expected that its Authority signals have migrated (with any dampening effect etc) across to PageB.html.
Link Explorer | | Infotex_UK0