I got a 404 in the Crawl Test Tool Report
-
I, yesterday i ran an crawl on http://www.everlastinggarden.nl and i get an 404. Does anybody know why this happens?
<colgroup><col width="1535"></colgroup>
| # ---------------------------------------- |
| Crawl Test Tool Report | Moz,http://pro.seomoz.org/tools/crawl-test |
| www.everlastinggarden.nl |
| Report created: 15 Jul 18:34 |
| # ---------------------------------------- |
| URL,Time Crawled,Title Tag,Meta Description,HTTP Status Code,Referrer,Link Count,Content-Type Header,4XX (Client Error),5XX (Server Error),Title Missing or Empty,Duplicate Page Content,URLs with Duplicate Page Content (up to 5),Duplicate Page Title,URLs with Duplicate Title Tags (up to 5),Long URL,Overly-Dynamic URL,301 (Permanent Redirect),302 (Temporary Redirect),301/302 Target,Meta Refresh,Meta Refresh Target,Title Element Too Short,Title Element Too Long,Too Many On-Page Links,Missing Meta Description Tag,Search Engine blocked by robots.txt,Meta-robots Nofollow,Blocked by X-robots,X-Robots-Tag Header,Blocked by meta-robots,Meta Robots Tag,Rel Canonical,Rel-Canonical Target,Blocking All User Agents,Blocking Google,Blocking Yahoo,Blocking Bing,Internal Links,Linking Root Domains,External Links,Page Authority |
| http://www.everlastinggarden.nl,2014,404 : Received 404 (Not Found) error response for page.,Error attempting to request page |Best regards,
Jos
-
Erin,
thank you very much for your effort. I already contacted my host on this problem, but they say is that they are only blocking Yandex and that every other crawler is allowed.
In the past my client did have similair errors (website not reachable) but that was solved i understood. I will contact you via in private for additional information.
Also I will contact the host again and ask them to investigate the matter.
Best regards, Jos
-
Hi Jos,
Thanks for reaching out! My name is Erin, and I'm on the Moz Help Team. Our crawls have been delayed, and we're currently working through the backlog. Thanks for your patience! Keep in mind that the Crawl Test is cached for 48 hours, so if you run them too close together you'll see the same data.
I curled your site from our servers, and I came back with a 404. Usually this means that your site is blocking us on the server level- and explains why your site comes back with a 200 from a Header Checker, but a 404 from our servers. I did a little digging but wasn't able to come up with much, so you'll want to loop in your web dev to see if they can shed some more light on what might be happening. I'm sorry there's not more I can do for you on the support side with this one!
I don't want to share any private site details over the interwebz, so if you have any follow up questions email the Help Team.
I hope that helps, and happy Monday!
Cheers,
Erin
-
Hi! I've asked our help team to come and take a look at this and investigate. Thanks for your patience!
-
Can anybody help? I re-issued another crawl, but this one is already 4 days on it's way ????
See attachement.
Best regards, Jos
-
Er_Maqui, Keri,
thank you for you responses. I informed with the host, the only block Yandex everything else is allowed.
I did a re-do of the crawl with the same result ;-(
@Keri, it's a 404 on the homepage.... :
http://www.everlastinggarden.nl,2014,404 : Received 404 (Not Found) error response for page.,Error attempting to request page.
Also in my campaign it gives a 404. If i check the a-record, all looks good: https://www.whatsmydns.net/#A/www.everlastinggarden.nl
Best regsrds, Jos
-
Hi! If you download the CSV of your crawl report, it will list the referring URL that is pointing to the page with a 404.
-
Try to re-do the crawl test. This can be a puntual problem.
For getting more info, check your server log. Try to enter the page before re-do the test for verify all it's working.
Also, if you can, check if your hosting is blocking some user-agents (question it to them) for reduce traffic.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Reports based on labels
Hi all, I'm trying to create a custom report using one of the templates, but I only want it to show keywords that have a specific label assigned to it. Is this possible? Thank you
Moz Bar | | David_Mo0 -
How do I pull a rankings report for a specific week in the past?
The date of the rankings I want is January 13th. I can see the individual keyword history in rank tracker, but I have to go each word. I'd like to just have that report to compare it to, since that is when a lot of our rankings changed. So I need that report to compare our future efforts with. Please advise and thank you! Ruben
Moz Bar | | KempRugeLawGroup0 -
Duplicate Home Page in Moz Report
I recently ran a crawl report in MOZ for www.laddersfree.co.uk and it has picked duplicate pages for my home page. The two pages are www.laddersfree.co.uk and www.laddersfree.co.uk The CMS System my developer is using is Concrete5. If you look at the source code there is a Canonical <link rel="<a class="attribute-value">canonical</a>" href="http://www.laddersfree.co.uk/" /> I am very confused as to why there are two pages picked up in the crawl. Can someone offer any advice please? Thank you. Jason
Moz Bar | | gymmad0 -
Why does the moz crawl test lists page twice?
Hi, I'm running into an issue where some crawlers list my pages twice, once with a trailing slash, once without. I first saw it on a few pages with screaming frog, then saw it happen on all my pages with the moz crawler. The site is www.kidsandart.org and its on Squarespace. I grepped the sitemap.xml I submitted to google webmaster and got 167 distinct pages, all of them without a trailing slash. Any insights on why this is happening, and how to regard moz crawler results would be appreciated. thanks Tom
Moz Bar | | tpushpathadam0 -
Moz Crawl Showing Duplicate Content But It's Not?!
Unfortunately I can't give out the URL, but here's the deal... I have two URL's which have completely different content on them but are being crawled as duplicate content. Any Idea how that would happen? I'm not seeing any errors in WMT's. Has anyone seen this before? Is the duplicate content reporting based on a % of the page content matching as the same?
Moz Bar | | Swarm-SEO0 -
Where did the Email Reports go?
Since the switch to the new interface, I can't find where the email reports are. I used to be able to select multiple metrics to build a custom report that would get sent to me each month, doesn't seem to be an option anymore. Where'd they go?
Moz Bar | | benmarshall0 -
Bing data in the keyword difficulty and SERP analysis tool
Hi Guys, I'm wondering about the exact match bing data in the Moz KWD & SERP analysis tool. We're looking at the ranking difficulty for given terms in Google but getting exact match bing numbers. Is the tool estimating Google searches from the bing data (perhaps a simple formula assuming maybe 10% search totals for Bing, 60% for Google or whatever is closest these days) or is it just raw bing numbers that we then have to upsize in order to estimate search numbers in Google? Hope that makes some kind of sense! In other words - how are you guys using that Bing data when considering terms to target using this tool? Look forward to hearing your thoughts.
Moz Bar | | lovealbatross
Cheers
James0 -
How do I get Moz to crawl my site?
If I'm not mistaken, Moz will crawl my site once a week and let me know of any errors, notices etc.. But can I ask them to crawl once I have updated all the errors? or do I have to wait for them to do this automatically?
Moz Bar | | david.smith.segarra0