Crawl Diagnostics Report Lacks Information
-
When I look at the crawl diagnostics, SEOMoz tells me there are 404 errors.
This is understandable, because some pages were removed.
What this report doesn't tell me is how those pages were discovered.
This is a very important piece of information, because it would tell me there are links pointing to those pages, either internal or external. I believe the internal links have been removed.
If the report told me how if found the link, I would be able to take immediate action. Without that information, I have to go so a lot of investigation. And when you have a million pages, that isn't easy.
Some possibilities:
- The crawler remembered the page from the previous crawl.
- There was a link from an index page - i.e. it is in the database still
- There was an individual link from another story - so now there are broken links
- Ditto, but it in on a static index page
- The link was from an external source - I need to make a redirect
Am I missing something, or is this a feature the SEO Moz crawler doesn't have yet?
What can I do (other than check all my pages) to discover this?
-
OK thank you, Ralph
I can work on that.
-
I think it's the SEOMoz crawler, but what I have found is that the error reports are limited here whereas GWT is much bigger and shows the links leading to the error. My guess is that SEOMoz limit the number of crawl errors they show due to limitations set on their crawler i.e. while their crawl is comprehensive, it's not going to capture what Google does.
-
Thank you Ralph.
Yes, had it for years. So is this a GWT report? I thought it was SEOMoz !
No not IIS, Linux.
-
If you download the csv file for the crawl you can sort it by http status to get all of the 404 errors together. Then there is a specific column that contains the referrer that provides the information you are after.
-
This may be a silly question, but have you got Google Webmaster tools installed? That will show you the source of the errors.
If your site is on IIS then you should also use the awesome IIS SEO toolkit provided by Microsoft for free.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Moz Pro crawl signaling missing canonical which are not?
Hi,
Moz Pro | | rolandvintners
I'm trying MozPro considering using it.
One of the tool which is appealing is the crawl and insights.
After quick use, I really question many of the alerts, for instance, I got a "missing canonical tag" on this url: https://vintners.co/wine/grawu_gto#2020 but when I check my markup, there's clearly a canonical tag: <link rel="canonical" href="https://vintners.co/wine/grawu_gto"> Anybody can explain?
I asked Moz Pro staff when being onboarded but didn't get an answer...
Honestly, I'm questioning the value of these crawls, or may be I miss something?0 -
SEOmoz crawler not crawling my site
We set up a new campaign in SEOmoz on Friday. It is my understanding that the preliminary crawl can cover up to 250 and this has been our experience in the past. However, the preliminary crawl only went through 2 pages. This is a larger eCommerce site with many pages. Any ideas why more pages weren't crawled? We set up the campaign to track at the root domain level.
Moz Pro | | IMM0 -
Issue with Opesiteexplorer report
Hi, When I pulled backlinks report for my website from opensite explorer. I see the backlinks listing for my homepage is duplicated with index page. Ex. For a external website my homepage www.website.com as well as www.website.com/index.php shows as backlinks. Note: I had given rel=canonical tag to my homepage and index.php is 301 redirection to the homepage. I find this error only to my homepage and I don't find this issue to my inner pages. I would like to know whether this is an issue with opensiteexplorer or a problem with my homepage url. Your answers are appreciated.
Moz Pro | | massimobrogi0 -
Site Ranking Report
Hi guys, My site ranking report says that I've gone from being 1-20 for a variety of keywords in Google UK to not in the top 50. When I do a search myself I see that my site remains where it previously was (between 1-20). How reliable is the site ranking reporting on a weekly basis? Is it best to look at it monthly?
Moz Pro | | columbus0 -
SEOMoz Crawl Warnings, do they really hurt rankings?
SEOMoz reports 250 crawl warnings on my site. In most cases its too long title tags, with 4 of them its missing meta description. SEOMoz says it will hurt my rankings? However, I'm sure a recent whiteboard Friday contradicted this. So what is it?
Moz Pro | | sanchez19600 -
Crawling a website with redirects
Hi, I started a campaign for a website which uses multiple redirects before showing the real content. in the crawling report only one page is crawled. Is there a way to let the crawler pass the redirects to get usefull reports? The website is www.cegeka.be Thank you
Moz Pro | | Cegeka0 -
How to handle crawl diagnostic errors for the same url. /products & /products/
I have copied on of the errors out of the crawl diagnostics report. Both /products and /products/ are returning an error, and both have pretty good domain authority so I feel like its hurting my site that these show up this way. Both urls create the same page, should I just setup a 301 on the /products with no slash or will that cause more harm... I am using the MODx cms system and that could have something to do with it. | Products | Datalight http://www.datalight.com/products 1 37 5 Products | Datalight http://www.datalight.com/products/ | 1 | 30 | 1 |
Moz Pro | | tjsherrill0 -
SEOmoz keyword rankings in campaign report
Hi. Does anybody know where the rankings summary has moved to in the campaign reports? I want to know how many keywords have moved up or down in the last week and can't find it anywhere!
Moz Pro | | neooptic1