Tracking a Crawl error
-
Hi All,
If you find a crawl error on your page. How do you find it?
The error only says the URL that is wrong but this is not the location. Can i drill down and find out more information?
Thank you!
-
Hi Martin,
Thanks for coming back to me.
You are spot on... I think I have been sitting at the desk to long today. Zoned out!
Yep, webtools in google shows the URL in full so I can find it now.
Thanks for your help
-
Hi Wayne, Just to clarify you are having issues with a crawl error in Webmaster tools, and when you click the url it works fine? You can use tools such as Google Webmaster tools > Fetch as Google bot and input the url to see how Google Bot see's it in HTML. You can also download a text based browser such as Lynx to view your website very similar to how Google bot see's it. It will be very hard to determine a Crawl Path, as if that information was available we could develop trends and get a little closer to their algorithm. Crawler trackers can, on occasion cause hinderence to the crawler.Hope this helps.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
404 crawl errors ending with your domain name??
Hello, I have a crawl test with numerous 404 errors ending with my domain name..? Not sure what the cause is. Plugins? Ecommerce? I use Wordpress if that could lead to an answer. Thanks for your time. K
Technical SEO | | Hydraulicgirl0 -
:443 - 404 error
I get strange :443 errors in my 404 monitor on Wordpress https://www.compleetverkleed.nl:443/hoed-al-capone-panter-8713647758068-2/
Technical SEO | | Happy-SEO
https://www.compleetverkleed.nl:443/cart/www.compleetverkleed.nl/feestkleding
https://www.compleetverkleed.nl:443/maskers/ I have no idea where these come from :S2 -
Homepage De-Indexed - No Errors, No Warnings
Hi, I am currently working on this project. Sometime between March 7th & 8th homepage was de-indexed. The rest of the pages are there. Found it out through decreased traffic on GA. No notifications of any kind of penalty/errors recieved. Tried to manually re-index through "Fetch as Google" in WMT to no avail. Site is redirected to https. Any suggestions would be highly appreciated. Thank you in advance.
Technical SEO | | gpapatheodorou0 -
How can I fix this home page crawl error ?
My website shows this crawl error => 612 : Home page banned by error response for robots.txt. I also did not get any page data in my account for this website ... I did get keyword rankings and traffic data, I am guessing from the analytics account. url = www.mississaugakids.com Not sure really what to do with this ! Any help is greatly appreciated.
Technical SEO | | jlane90 -
HTTP 500 Internal Server Error, Need help
Hi, For a few days know google crawlers have been getting 500 errors from our dedicated server whenever they try to crawl the site. Using the "Fetch as Google" tool under health in webmaster tools, I get "Unreachable page" every time I fetch the homepage. Here is exactly what the google crawler is getting: <code>HTTP/1.1 500 Internal Server Error Date: Fri, 21 Jun 2013 19:52:27 GMT Server: Apache/2.2.15 (CentOS) X-Powered-By: PHP/5.3.3 X-Pingback: [http://www.communityadvocate.com/xmlrpc.php](http://www.communityadvocate.com/xmlrpc.php) Connection: close Transfer-Encoding: chunked Content-Type: text/html; charset=UTF-8 http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> My url is [http://www.communityadvocate.com](http://www.communityadvocate.com/)</code> and here's the screenshot from Goolge webmater http://screencast.com/t/FoWvqRRtmoEQ How can i fix that? Thank you
Technical SEO | | Vmezoz0 -
Help! Getting 5XX error
Keep getting a 5XX error and my site is obviously losing ranking, Asked the hoster. Nobody seems to know what is wrong. Site is www.monteverdetours.com I know this is probably an obvious problem and easy to fix but I don't know how to do it! Any comments will be greatly appreciated.
Technical SEO | | Llanero0 -
Crawl rate
Hello, In google WMT my site has the following message. <form class="form" action="/webmasters/tools/settings-ac?hl=en&siteUrl=http://www.prom-hairstyles.org/&siteUrl=http://www.prom-hairstyles.org/&hl=en" method="POST">Your site has been assigned special crawl rate settings. You will not be able to change the crawl rate.Why would this be?A bit of backgound - this site was hammered by Penguin or maybe panda but seems to be dragging itself back up (maybe) but has dropped from several thousand visitors/day to 100 or so.Cheers,Ian</form>
Technical SEO | | jwdl0 -
Summarize your question.Crawl Diagnostics Summary
Hi, Crawl Diagnostics Summary pointed on some mistakes I've done, I fixed them, but Crawl Diagnostics Summary still shows same errors, how often does ithe data refreshes?
Technical SEO | | AndreyStotsky0