How Is a Page Crawled by Moz When Moz Says 'No Links'?
-
As above, really. I've crawled a new client's site to find the Moz crawler has identified a handful of 404 errors.
The Moz crawler says these pages have '0 linking domains', and OSE has no data for these pages.
So how are these pages being crawled by Moz and what should I advise my client?
-
Thanks again, Peter.
-
That's great, thanks Keri.
-
Hi, no problem. Keri has answered your query re the Moz crawler not finding them.
Peter
-
If you download the CSV report, we list the referring page there. We know it isn't obvious and we're working on making this a little easier to find.
-
Hi Keri,
Thanks for getting back to me (thanks to Peter above, also).
So what is the best way of me finding where these 404 pages are linked from?
I am 99% sure they are internal links that I will ask the client to remove, but would like to avoid manually checking every link on the site if possible.
-
Hi Peter,
I searched for their domain name and about the expected number of results came up.
I searched for these individual URLs that Moz is reporting using the site: command, and they were not returned.
They are also neither in the site's sitemap (HTML page or XML file).
So how is SEOmoz Crawler finding them?
-
We have two completely different crawling systems. We have on crawler for your website, where you enter in your URL and we crawl your site looking for problems. We have a different crawler for OSE that goes and looks at the web in general and crawls for links. The two crawlers do not talk to each other, so it is actually understandable that OSE may not have data on the site. OSE doesn't have the same index size as Google, so we don't see all the links that are actually out there (though we're working all the time on improving this!).
-
Hi, if you put site:xxx.com into Google where xxx.com is your client's domain name, does Google report anything?
Peter
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
New Links Indexing Issue
Respected Team, I am facing new links indexing issue for one of my site, Is there any known indexing issue which is already addressed by Moz? Please help.
Link Explorer | | red-dr50 -
Fastest way to get new backlinks recognised by Moz?
Hi all, I have seen a few discussions / advice pieces around this, but wondered what the current view is on the fastest way to get new backlinks recognised by Moz? I have seen people say that the content needs to be shared (e.g. on Twitter), and advice around creating Bitly links. I am not convinced that these techniques are working. On a related note, my website is very large. I have around 2.5m pages. Most of these pages are 3-4 clicks from the home page, but I have a small amount of important content much higher up in the site (1-2 clicks from the home page). Does Moz crawl pages nearer the home page more frequently, or does it make a pass of all pages on the site before it starts at the beginning again? If it makes a complete pass of all pages, then presumably making my site smaller will help to get Moz to see new content more quickly? Is there any other value to be gained in making a large site smaller? Many thanks in advance, Neil.
Link Explorer | | NHARRIS10 -
How to interpret a possible spam link?
Hallo, Spam analysis is suggesting that my website may have a spam problem (score is 5). When I analyse the single spam links, I am puzzled. the links to my site are actually links in disqus comments I wrote myself on the linking site, in order to aswer REAL question of real people giving them honest and useful answers (and linking my site :-), like the link pointed out in the attached image, also pasted herebelow: http://www.androidiani.com/news/servizi-premium-indesiderati-lantitrust-contro-tim-vodafone-wind-e-3-213235 Most links have this same nature. Has this behaviour (which I consider natural, in good faith) been regarded as spammy? Should avoid linking my site in the comments I write on external sites? the links from comunicatistampagratis.it (towards the end of the list) are from a free press release website: you send them the press release, they publish it. Should I avoid using this free press release site? Should I avoid using any free press release site at all (since they are regarded as spammy)? is there a rule to learn (since apparently I have not been very good so far at using my good faith to separate good links from spammy links) whether a link can be spammy and therefore try and avoid it? thank you for you help.
Link Explorer | | micvitale0 -
When moz will index my page?
HI. I use moz pro for 1.5 month and yet my site is not indexed and i do not see Competitive Link Metrics. Can you please include meko.lv in index?
Link Explorer | | Mekounko0 -
No Internal Links
I have noticed on both OSE and also on the Crawl Test that our site www.advantageservices.net is showing only 1 internal link. I tried looking around as what might cause this but haven't been able to figure it out.
Link Explorer | | Kristine-ASNET0 -
Learn how to use Open Site Explorer's Just-Discovered Links report to build links and manage your brand. Get your Daily SEO Fix!
The Just-Discovered Links report in Open Site Explorer helps you discover recently created links--within an hour of them being published. It's pretty nifty. In today's Daily SEO Fix, Nick shows you how to use the report to view who is linking to you, how they're doing it, and what they are saying so you can capitalize on link opportunities while they're still fresh plus, monitor and participate in the conversations happening about your brand. Watch our tutorial Using Open Site Explorer to Find Fresh Links and Manage Your Brand Online and get ready to dive in! This video is part of The Moz Daily SEO Fix tutorial series--Moz tool tips and tricks in under 2 minutes. To watch all of our videos so far, and to subscribe to future ones, make sure to visit the Daily SEO Fix channel on YouTube.
Link Explorer | | kellyjcoop3 -
Do 'Just Discovered' Links get added to the main link index?
Hi, I was wondering if the 'Just Discovered' links get added to the main link crawl index? It would seem to make sense for them to do so, as this would enable the enable the link index to be more up to date than it would otherwise be. Observing the link index it would seem that at the moment it does not do this and they are totally separate indexes (based on personal observation). Thanks
Link Explorer | | James770