I got a 404 in the Crawl Test Tool Report
-
I, yesterday i ran an crawl on http://www.everlastinggarden.nl and i get an 404. Does anybody know why this happens?
<colgroup><col width="1535"></colgroup>
| # ---------------------------------------- |
| Crawl Test Tool Report | Moz,http://pro.seomoz.org/tools/crawl-test |
| www.everlastinggarden.nl |
| Report created: 15 Jul 18:34 |
| # ---------------------------------------- |
| URL,Time Crawled,Title Tag,Meta Description,HTTP Status Code,Referrer,Link Count,Content-Type Header,4XX (Client Error),5XX (Server Error),Title Missing or Empty,Duplicate Page Content,URLs with Duplicate Page Content (up to 5),Duplicate Page Title,URLs with Duplicate Title Tags (up to 5),Long URL,Overly-Dynamic URL,301 (Permanent Redirect),302 (Temporary Redirect),301/302 Target,Meta Refresh,Meta Refresh Target,Title Element Too Short,Title Element Too Long,Too Many On-Page Links,Missing Meta Description Tag,Search Engine blocked by robots.txt,Meta-robots Nofollow,Blocked by X-robots,X-Robots-Tag Header,Blocked by meta-robots,Meta Robots Tag,Rel Canonical,Rel-Canonical Target,Blocking All User Agents,Blocking Google,Blocking Yahoo,Blocking Bing,Internal Links,Linking Root Domains,External Links,Page Authority |
| http://www.everlastinggarden.nl,2014,404 : Received 404 (Not Found) error response for page.,Error attempting to request page |Best regards,
Jos
-
Erin,
thank you very much for your effort. I already contacted my host on this problem, but they say is that they are only blocking Yandex and that every other crawler is allowed.
In the past my client did have similair errors (website not reachable) but that was solved i understood. I will contact you via in private for additional information.
Also I will contact the host again and ask them to investigate the matter.
Best regards, Jos
-
Hi Jos,
Thanks for reaching out! My name is Erin, and I'm on the Moz Help Team. Our crawls have been delayed, and we're currently working through the backlog. Thanks for your patience! Keep in mind that the Crawl Test is cached for 48 hours, so if you run them too close together you'll see the same data.
I curled your site from our servers, and I came back with a 404. Usually this means that your site is blocking us on the server level- and explains why your site comes back with a 200 from a Header Checker, but a 404 from our servers. I did a little digging but wasn't able to come up with much, so you'll want to loop in your web dev to see if they can shed some more light on what might be happening. I'm sorry there's not more I can do for you on the support side with this one!
I don't want to share any private site details over the interwebz, so if you have any follow up questions email the Help Team.
I hope that helps, and happy Monday!
Cheers,
Erin
-
Hi! I've asked our help team to come and take a look at this and investigate. Thanks for your patience!
-
Can anybody help? I re-issued another crawl, but this one is already 4 days on it's way ????
See attachement.
Best regards, Jos
-
Er_Maqui, Keri,
thank you for you responses. I informed with the host, the only block Yandex everything else is allowed.
I did a re-do of the crawl with the same result ;-(
@Keri, it's a 404 on the homepage.... :
http://www.everlastinggarden.nl,2014,404 : Received 404 (Not Found) error response for page.,Error attempting to request page.
Also in my campaign it gives a 404. If i check the a-record, all looks good: https://www.whatsmydns.net/#A/www.everlastinggarden.nl
Best regsrds, Jos
-
Hi! If you download the CSV of your crawl report, it will list the referring URL that is pointing to the page with a 404.
-
Try to re-do the crawl test. This can be a puntual problem.
For getting more info, check your server log. Try to enter the page before re-do the test for verify all it's working.
Also, if you can, check if your hosting is blocking some user-agents (question it to them) for reduce traffic.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Http:// https:// google search console crawl errors
How to direct http:// to https:// to get rid of 404 errors in google webmaster search console (http:// crawl errors)
Moz Bar | | O.D.0 -
Does Moz's keyword tool pull data from your IP address?
Does anyone know how Moz's keyword tool pulls their keyword ranks? Do they take it based off of the IP (history and cookies) that is being used? I am trying to find a way to collect keyword data that is neutral and not based off of my previous searches, etc. TIA
Moz Bar | | ReviveMedia0 -
Duplicate Home Page in Moz Report
I recently ran a crawl report in MOZ for www.laddersfree.co.uk and it has picked duplicate pages for my home page. The two pages are www.laddersfree.co.uk and www.laddersfree.co.uk The CMS System my developer is using is Concrete5. If you look at the source code there is a Canonical <link rel="<a class="attribute-value">canonical</a>" href="http://www.laddersfree.co.uk/" /> I am very confused as to why there are two pages picked up in the crawl. Can someone offer any advice please? Thank you. Jason
Moz Bar | | gymmad0 -
How can I find the old ERRORS and WARNINGS report in the NEW Moz design?
I'm looking for a complete list of errors and bugs that need to be fixed within a website. I used to use the MAIN tool (at least it seemed it was the most popular) but now that its just MOZ.com I can't seem to find that great report. It had data such as: 1. List of pages with Title Tags too long 2. List of pages with Description Tags too long 3. List of RED errors and YELLOW warnings, BLUE somethings... etc... Ring a bell? I LOVED this report, where can I find this data? Thanks! Derek
Moz Bar | | DerekM42420 -
Weird back link showed in moz crawl
Some time ago somebody from this site: http://dianibeach.com created a weird link to our site which had on the end db. Later we have realized that the link was coming from every footer on each page. I believe that the back links from footer does not have realy value and even the more of them the less value. We have asked the guy to remove that links as I thought it might harm our site more then help. Now I I was very surprised to find this link in moz crawl error as second top page on our site in current index??? Can somebody explain how is this possible?? The most ridiculous thing is that when I click on that link it realy opens our site! How is that possible, what is it? This is the link: http://villasdiani.com/?db Thank you very much for any help with this
Moz Bar | | Rebeca10 -
On Page reporting Issue
Hi I’ve recently downloaded my onpage analysis report for our site http://www.health2000.co.nz . The column that is titled “Missing data description Tag” says false, which I’m assuming means the meta description is present. However in the column titled “meta _description tag” there is no meta description present (nor is any meta description in the backend of my site. Could someone please explain why this would occur? I look forward to hearing from you Regards Pete
Moz Bar | | Hardley1110 -
Is Manual Crawl Test option available now to Pro Users?
Hi all, I have worked on my Crawl Issues and want to see how many still exist. Earlier I was using Manual Crawl Test. However, now I don't see this tool in Moz Account. Please suggest. Thanks
Moz Bar | | chandman0 -
Where is the SEOmoz Term Extractor tool, I am not seeing it under Research and Tools
Where is the SEOmoz Term Extractor tool, I am not seeing it under Research and Tools
Moz Bar | | dgodbold0