Site Crawl 4xx Errors?
-
Hello!
When I check our website's critical crawler issues with Moz Site Crawler, I'm seeing over 1000 pages with a 4xx error.
All of the pages that are showing to have a 4xx error appear to be the brand and product pages we have on our website, but with /URL at the end of each permalink.
For example, we have a page on our site for a brand called Davinci. The URL is https://kannakart.com/davinci/. In the site crawler, I'm seeing the 4xx for this URL: https://kannakart.com/davinci/URL.
Could this be a plugin on our site that is generating these URLs? If they're going to be an issue, I'd like to remove them. However, I'm not sure exactly where to begin.
Thanks in advance for the help,
-Andrew
-
Paul,
Makes perfect sense. Thank you for the clear answer and explanation!
-
You have a defective link coded for the FAQ link in your site's footer, Andrew. It's currently coded as
[Which then gets parsed as a relative link. This means the link gets parsed as www.example.com/currentpageurl/URL
Because that link is in the sitewide footer, it means every page is generating that link as a link to itself with the /URL added to the end. Which you're seeing detected as all those 404s.
This is very important to fix not only because the FAQ link is broken in the footer, but you've doubled the number of pages of the site for the search crawlers, meaning they're wasting time following thousands of useless links instead of focusing on real pages.
So fortunately it's a quick fix - find that link in your footer template and correct it.
Make sense?
Paul](URL)
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Does SEOmoz have a tool to find mirror sites?
I heard from a company that is trying to get my clients SEO business that they discovered multiple sites mirroring our site's content. Does SEOmoz have a tool to find these websites? Or does Google?
Moz Pro | | thomas.wittine0 -
How effective is Crawl DIagnostics in determining crawlibility?
Is Seomoz crawl diagnostics useful for determining what pages Google has a hard time indexing. One of the problems with my site is that it uses JS and Flash and I know Google isnt too keen on that. Can Crawl Diagnostics accurately tell me if there is too much of something and therefore Google is having a hard time crawling? I want to be able to know if JS or Flash is hurting any of my pages in any way. I provide good content and I want to make sure Google can pick it up.....Is this too much to ask? Is there anything out there for this?
Moz Pro | | waltergah0 -
A question about Mozbot and a recent crawl on our website.
Hi All, Rogerbot has been reporting errors on our website's for over a year now, and we correct the issues as soon as they are reported. However I have 2 questions regarding the recent crawl report we got on the 8th. 1.) Pages with a "no-index" tag are being crawled by roger and are being reported as duplicate page content errors. I can ignore these as google doesnt see these pages, but surely roger should ignore pages with "no-index" instructions as well? Also, these errors wont go away in our campaign until Roger ignores the URL's. 2.) What bugs me most is that resource pages that have been around for about 6 months have only just been reported as being duplicate content. Our weekly crawls have never picked up these resources pages as being a problem, why now all of a sudden? (Makes me wonder how extensive each crawl is?) Anyone else had a similar problem? Regards GREG
Moz Pro | | AndreVanKets0 -
How does SEOMoz crawl sites? Does it follow the sitemap?
I only ask because in my report it returned a few "no follow" links. Was wondering how SEOMoz operates.
Moz Pro | | Pikimal0 -
Moz crawling
Hi Everyone! I'm new to the SEOMoz and wanted to find out if there is a way to decrease the waiting time for the campaign crawl. I have made a lot of changes based on the first crawl and would like to see how these are reflected on the reports, but can't until the next crawl is performed. Any help would be greatly appreciated.
Moz Pro | | coremediadesign0 -
Crawling One Page
I set up a profile for a site with many pages, opting for setting up as a root directory. When SEOMoz crawled, they only found one page. Any ideas for why this would be? Thanks!
Moz Pro | | Group160 -
Canonical tags and SEOmoz crawls
Hi there. Recently, we've made some changes to http://www.gear-zone.co.uk/ to implement canonical tags to some dynamically generated pages to stop duplicate content issues. Previously, these were blocked with robots.txt. In Webmaster Tools, everything looks great - pages crawled has shot up, and overall traffic and sales has seen a positive increase. However the SEOmoz crawl report is now showing a huge increase in duplicate content issues. What I'd like to know is whether SEOmoz registers a canonical tag as preventing a piece of duplicate content, or just adds to it the notices report. That is, if I have 10 pages of duplicate content all with correct canonical tags, will I still see 10 errors in the crawl, but also 10 notices showing a canonical has been found? Or, should it be 0 duplicate content errors, but 10 notices of canonicals? I know it's a small point, but it could potentially have a big difference. Thanks!
Moz Pro | | neooptic0