How Is a Page Crawled by Moz When Moz Says 'No Links'?
-
As above, really. I've crawled a new client's site to find the Moz crawler has identified a handful of 404 errors.
The Moz crawler says these pages have '0 linking domains', and OSE has no data for these pages.
So how are these pages being crawled by Moz and what should I advise my client?
-
Thanks again, Peter.
-
That's great, thanks Keri.
-
Hi, no problem. Keri has answered your query re the Moz crawler not finding them.
Peter
-
If you download the CSV report, we list the referring page there. We know it isn't obvious and we're working on making this a little easier to find.
-
Hi Keri,
Thanks for getting back to me (thanks to Peter above, also).
So what is the best way of me finding where these 404 pages are linked from?
I am 99% sure they are internal links that I will ask the client to remove, but would like to avoid manually checking every link on the site if possible.
-
Hi Peter,
I searched for their domain name and about the expected number of results came up.
I searched for these individual URLs that Moz is reporting using the site: command, and they were not returned.
They are also neither in the site's sitemap (HTML page or XML file).
So how is SEOmoz Crawler finding them?
-
We have two completely different crawling systems. We have on crawler for your website, where you enter in your URL and we crawl your site looking for problems. We have a different crawler for OSE that goes and looks at the web in general and crawls for links. The two crawlers do not talk to each other, so it is actually understandable that OSE may not have data on the site. OSE doesn't have the same index size as Google, so we don't see all the links that are actually out there (though we're working all the time on improving this!).
-
Hi, if you put site:xxx.com into Google where xxx.com is your client's domain name, does Google report anything?
Peter
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Sudden surge in discovered links
Hi everyone, Recently Moz discovered a lot of old backlinks to my url. Some of them don't even exist anymore. I don't understand how Moz has found these links, even if they don't exist? And why did Moz find them suddenly all at once? Can anyone help me with this question? Thanks!
Link Explorer | | Online_Marketing_Monkey0 -
How to index backlinks in moz?
hello my website has some backlinks from high authority website that has been indexed by google but moz is not indexing my backlinks and also my pages so what should i do?
Link Explorer | | yuewiioweiwjksa38320 -
Incorrect crawl errors
A crawl of my websites has indicated that there are some 5XX server errors on my website: Error Code 608: Page not Decodable as Specified Content Encoding
Link Explorer | | LiamMcArthur
Error Code 803: Incomplete HTTP Response Received
Error Code 803: Incomplete HTTP Response Received
Error Code 608: Page not Decodable as Specified Content Encoding
Error Code 902: Network Errors Prevented Crawler from Contacting Server The five pages in question are all in fact perfectly working pages and are returning HTTP 200 codes. Is this a problem with the Moz crawler?1 -
Can't make sense of OSE and MOZ.
I checked this site on my OSE and it shows only 7 inbound links a DA of 14 and no social activity whatsoever yet when I check it on Majestic it shows ExternalBacklinks 71 ReferringDomains 20 Referring IPs 19 Referring Subnets 19 . And.. the page is #1 on google search for hypnotherapy michigan. How can it rate so poorly on MOZ, show so much more on majestic and rank so high on google? I thought MOZ data was supposed to be among the best and that top rated pages on google should also rate high on MOZ. here is the site http://hypnotherapy-detroit.com Additionally, when i look at the site, i notice that most of the backlinks are exchanged links and this site's link exchange page isn't even linked from the home page. Now I thought that kind of link exchange game was now discounted by Google. I don't get it. No social pages at all... low page rank... no new content.. so by MOZ standards there is no justification for this page to be anywhere near page one let alone at position #1. Can someone help me make sense of all this?
Link Explorer | | HypnoPro0 -
Learn how to use Moz's Link Intersect tool to build your link profile. Get your Daily SEO Fix!
The quantity and (more importantly) quality of backlinks to your website make up your link profile; one of the most important elements in SEO and an incredibly important factor in search engine rankings. In today's Daily SEO Fix, How to Use Link Intersect to Build Your Link Profile, Tori shows you how to use Moz's Link Intersect tool to analyze the competitions' backlinks plus, find opportunities to build links and strengthen your own link profile. This video is one of our last videos in The Moz Daily SEO Fix tutorial series--Moz tool tips and tricks in under 2 minutes. To watch all of our videos so far, make sure to visit the Daily SEO Fix channel on YouTube. And, if you have suggestions for future tutorials, please let us know via the comments below. Thanks!
Link Explorer | | kellyjcoop2 -
Feedburner Links Showing Up In OSE
I, for years now, have been seeing lots of links from http://feeds.feedburner.com/ showing up very high in backlink profiles. Why is this? And if these links are not passing link juice why are they being reported on in OSE? Seems like there should be some sort of filter for these, or that OSE bot should follow robots.txt directives as googlebot does.
Link Explorer | | MiguelSalcido0 -
Advanced Inbound Links Report - hanging at 99,450?
Hi, the rest of my report seemed to process quickly, but now it's at 99,450,
Link Explorer | | matthaslem
it doesn't seem to be processing further. It's been about 12 hours since requesting the report, any ideas? Thanks!0 -
Links from no-indexed pages in Open Site Explorer
I just looked up a site in Open Site Explorer. After putting in the site, I get a list of domains, links etc which is very useful. At the bottom is a note which I have pasted below. It mentions "24 links from no-indexed pages" and says we can't get any info on these. Is there a way to see what the 24 links are so I can investigate further?"Why does my number of inbounds links listed not match my total links count? The listed number may be slightly different because we only show 25 links per domain (so that you see a variety). Additionally you have 24 links from no-indexed pages. Unfortunately, we can't get info for those links."
Link Explorer | | EdKim0