I got a 404 in the Crawl Test Tool Report
-
I, yesterday i ran an crawl on http://www.everlastinggarden.nl and i get an 404. Does anybody know why this happens?
<colgroup><col width="1535"></colgroup>
| # ---------------------------------------- |
| Crawl Test Tool Report | Moz,http://pro.seomoz.org/tools/crawl-test |
| www.everlastinggarden.nl |
| Report created: 15 Jul 18:34 |
| # ---------------------------------------- |
| URL,Time Crawled,Title Tag,Meta Description,HTTP Status Code,Referrer,Link Count,Content-Type Header,4XX (Client Error),5XX (Server Error),Title Missing or Empty,Duplicate Page Content,URLs with Duplicate Page Content (up to 5),Duplicate Page Title,URLs with Duplicate Title Tags (up to 5),Long URL,Overly-Dynamic URL,301 (Permanent Redirect),302 (Temporary Redirect),301/302 Target,Meta Refresh,Meta Refresh Target,Title Element Too Short,Title Element Too Long,Too Many On-Page Links,Missing Meta Description Tag,Search Engine blocked by robots.txt,Meta-robots Nofollow,Blocked by X-robots,X-Robots-Tag Header,Blocked by meta-robots,Meta Robots Tag,Rel Canonical,Rel-Canonical Target,Blocking All User Agents,Blocking Google,Blocking Yahoo,Blocking Bing,Internal Links,Linking Root Domains,External Links,Page Authority |
| http://www.everlastinggarden.nl,2014,404 : Received 404 (Not Found) error response for page.,Error attempting to request page |Best regards,
Jos
-
Erin,
thank you very much for your effort. I already contacted my host on this problem, but they say is that they are only blocking Yandex and that every other crawler is allowed.
In the past my client did have similair errors (website not reachable) but that was solved i understood. I will contact you via in private for additional information.
Also I will contact the host again and ask them to investigate the matter.
Best regards, Jos
-
Hi Jos,
Thanks for reaching out! My name is Erin, and I'm on the Moz Help Team. Our crawls have been delayed, and we're currently working through the backlog. Thanks for your patience! Keep in mind that the Crawl Test is cached for 48 hours, so if you run them too close together you'll see the same data.
I curled your site from our servers, and I came back with a 404. Usually this means that your site is blocking us on the server level- and explains why your site comes back with a 200 from a Header Checker, but a 404 from our servers. I did a little digging but wasn't able to come up with much, so you'll want to loop in your web dev to see if they can shed some more light on what might be happening. I'm sorry there's not more I can do for you on the support side with this one!
I don't want to share any private site details over the interwebz, so if you have any follow up questions email the Help Team.
I hope that helps, and happy Monday!
Cheers,
Erin
-
Hi! I've asked our help team to come and take a look at this and investigate. Thanks for your patience!
-
Can anybody help? I re-issued another crawl, but this one is already 4 days on it's way ????
See attachement.
Best regards, Jos
-
Er_Maqui, Keri,
thank you for you responses. I informed with the host, the only block Yandex everything else is allowed.
I did a re-do of the crawl with the same result ;-(
@Keri, it's a 404 on the homepage.... :
http://www.everlastinggarden.nl,2014,404 : Received 404 (Not Found) error response for page.,Error attempting to request page.
Also in my campaign it gives a 404. If i check the a-record, all looks good: https://www.whatsmydns.net/#A/www.everlastinggarden.nl
Best regsrds, Jos
-
Hi! If you download the CSV of your crawl report, it will list the referring URL that is pointing to the page with a 404.
-
Try to re-do the crawl test. This can be a puntual problem.
For getting more info, check your server log. Try to enter the page before re-do the test for verify all it's working.
Also, if you can, check if your hosting is blocking some user-agents (question it to them) for reduce traffic.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Moz reporting for C-Blocking
Hey Mozers, I see Moz has a reporting tool for C-blocking and for november I had 330. Does this mean 330 Ip addresses came from the same location in the month of november?
Moz Bar | | rpaiva1 -
Moz Crawl Test says pages have no internal links
Greetings, I am working on a website, https://www.nasscoinc.com, and ran a Moz Crawl Test on it. According to the crawl test, only 2 of the website's hundreds of pages are receiving internal links. When I run a similar test on the site using Screaming Frog, I see that most of the pages have at least one internal link. I'm wondering if anyone has seen this before with the crawl test; and there is a way to get the crawl test to see the internal links? Thanks!
Moz Bar | | TopFloor0 -
Scheduled Custom Reports Not Running
I have about 10 campaigns and 5 custom reports scheduled to run monthly for each campaign. Most of them have been running for years without issue, however, this month the majority have not run. There is an option to download past reports, yet the August 1st date is not highlighted. Anyone else having this issue or have a solution? There is one exception to this, a new campaign that had custom reports made with this design. They have a "action" button rather than "download past reports. Within this drop down is a option to download past PDFs. Is there a way this can just be downloaded without opening, like the previous reports?
Moz Bar | | Your_Workshop0 -
Moz ranking report shows massive loss in Bing SERP's. But it is not the case?
My weekly ranking report shows massive loss of SERP's on Bing. For most keywords I was no longer in the top 50 which where first top 5. But when I checked it out for myself, I'm still ranked top 5 for these keywords. I then used the Moz Rank Tracker and there it also says I'm still ranking top 5 for these keywords. What should I believe now?
Moz Bar | | wellnesswooz0 -
Optimise your Pages in Moz Crawl - where do the keywords come from?
I am just going through my first Crawl stats from the MOZ analytics Under the pages to optimise section I have pages that I have optimised for my best keyword with an A grade that are showing as an F grade and suggesting a different keyword? Where is this keyword coming from? I am assuming that my page has been analysed and a better keyword has been recommended? Can anyone advise? Thanks Roger
Moz Bar | | rnperki0 -
Moz "Crawl Diagnostics" doesn't respect robots.txt
Hello, I've just had a new website crawled by the Moz bot. It's come back with thousands of errors saying things like: Duplicate content Overly dynamic URLs Duplicate Page Titles The duplicate content & URLs it's found are all blocked in the robots.txt so why am I seeing these errors?
Moz Bar | | Vitalized
Here's an example of some of the robots.txt that blocks things like dynamic URLs and directories (which Moz bot ignored): Disallow: /?mode=
Disallow: /?limit=
Disallow: /?dir=
Disallow: /?p=*&
Disallow: /?SID=
Disallow: /reviews/
Disallow: /home/ Many thanks for any info on this issue.0 -
Is there any tool that checks the rankings at any time I want?
My problem is that in Moz when I check ranking it shows me only on a weekly basis. Acording to MOZ I was on rank 11 yesterday. This is not what my SEO guy has sent me in his report. And usually he is right. Would be nice to have a ranking report for each day. What is the sense of a ranking tool if I don't know what happened between 2 ranking dates? How can I check that? Is there any rank history tool? Or a tool that will check in real time. The way I do it sometimes is to go on google.com/ncr to see the us version of google because I am in Europe. Rank tracker does not work for me at the moment. Get ranking button is inactive. My old keyword list has not been imported from SEOMOZ
Moz Bar | | Valentinopolus0