How Is a Page Crawled by Moz When Moz Says 'No Links'?
-
As above, really. I've crawled a new client's site to find the Moz crawler has identified a handful of 404 errors.
The Moz crawler says these pages have '0 linking domains', and OSE has no data for these pages.
So how are these pages being crawled by Moz and what should I advise my client?
-
Thanks again, Peter.
-
That's great, thanks Keri.
-
Hi, no problem. Keri has answered your query re the Moz crawler not finding them.
Peter
-
If you download the CSV report, we list the referring page there. We know it isn't obvious and we're working on making this a little easier to find.
-
Hi Keri,
Thanks for getting back to me (thanks to Peter above, also).
So what is the best way of me finding where these 404 pages are linked from?
I am 99% sure they are internal links that I will ask the client to remove, but would like to avoid manually checking every link on the site if possible.
-
Hi Peter,
I searched for their domain name and about the expected number of results came up.
I searched for these individual URLs that Moz is reporting using the site: command, and they were not returned.
They are also neither in the site's sitemap (HTML page or XML file).
So how is SEOmoz Crawler finding them?
-
We have two completely different crawling systems. We have on crawler for your website, where you enter in your URL and we crawl your site looking for problems. We have a different crawler for OSE that goes and looks at the web in general and crawls for links. The two crawlers do not talk to each other, so it is actually understandable that OSE may not have data on the site. OSE doesn't have the same index size as Google, so we don't see all the links that are actually out there (though we're working all the time on improving this!).
-
Hi, if you put site:xxx.com into Google where xxx.com is your client's domain name, does Google report anything?
Peter
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Inbound Links Sudden Decline
Hi,
Link Explorer | | Jane090987
There is a question of my site that the number of linking domains has experienced a sudden decline - lost inbound links for around 100k in several days.
We are wondering the reason of this sudden decline, is it a common fluctation or is there some problems in our site? Thanks for your answers in advance!0 -
Internal Links is reporting "0" value
Open Site Explorer is reporting "0" value under "Total Internal Links" as well as "Internal Links" metrics for my website. I have a good interlinking of pages throughout the site.
Link Explorer | | anupjain
Is there an issue with OSE (Open Site Explorer) crawler ? OR There are some crawling problems with our Website's structure ?
Can anybody please give an idea or recommendations on this ?
Domain: https://www.traveltoindia.org/0 -
Moz bot transforms special characters into fake 404 Pages in Reports
I have a lot of 404 links in my exported Moz Pro reports which are not actually Not Found pages when checking the Referring URL. For example: Referring URL:
Link Explorer | | ProspectoGroup
https://ofertolino.ro/oferte/barn%C3%A4ngen 404:
https://ofertolino.ro/barn%C3%83%C2%A4ngen What happens is that the referring URL leads to a redirect (https://ofertolino.ro/barn%C3%A4ngen) but it never leads to the specified 404 Page (which has even a more weird symbol in the URL when it is url decoded) in the report. What could be the issue here and is the Moz bot responsible for such misbehaviour?0 -
Give a Solution my DA is not increasing and not index my backlinks in MOZ??
DA of my site is not increasing. It seems like Moz is not indexing my backlinks. Multiple backlinks of my site [ Tech Spotty ] is not indexing in Moz that is why DA is not increasing. Backlinks of my site have indexed in both Ahref and Semrush both not showing in Moz. DR has increased to 51 but DA is still 7. If anyone has the solution please let me know.
Link Explorer | | fbowable1 -
What's the easiest way to see a list of domains ranking well for a keyword with its DA and traffic numbers?
Hi. I'm trying to figure out how to easily see, for a given keyword, the ranking domains, their domain authority, and their traffic. Is it possible to get this all on one screen?
Link Explorer | | scott.thomason.scott80350 -
Inbound links for https://www.example.com versus just example.com
Hi all, Wondering the difference between the Inbound Links stats for https://www.example.com versus only using example.com. When I type in the https:// version, I see 3 Spam Flags - for the example.com version, there are none. There are also spammy links pointing to the https version as opposed to the non-https version when I use this tool: https://moz.com/researchtools/ose/ Anyone know the difference and which should be regarded when fixing spam issues?
Link Explorer | | netamorphosis0 -
Crawling and Links - Showing Links
Hello I have a campaign in Moz that shows a new site as having no domain authority and no links. The site does have links to it, but they don't show up on OSE. Does Moz count links in the comments section of blogs (or is this counted as social, when the commenting system is one that requires facebook login?) I know blog comments are not great for SEO, I just use them when the post/link is 100% genuinely relevant. The site does have actual links on websites not just comment links, so it isn't showing any type of link. What is the best way to see the links the site has? Thanks!
Link Explorer | | wearehappymedia1