How can I fix this home page crawl error ?
-
My website shows this crawl error =>
612 : Home page banned by error response for robots.txt.
I also did not get any page data in my account for this website ...
I did get keyword rankings and traffic data, I am guessing from the analytics account.
url = www.mississaugakids.com
Not sure really what to do with this !
Any help is greatly appreciated.
-
Hi there,
Is this still happening, or does it seem to have been taken care of?
Cheers,
Jane
-
Or just noindex it for now? Seems worth a try unless someone more technical has a better suggestion.
Are you seeing the error only when Moz crawls the site? Google Webmaster Tools isn't signaling any problems?
D
-
Thank you Donna,
That's pretty much what I had found, but did not think that would cause the home page error I am seeing. I know that events calendar is a problem for speed. Maybe I will move the calendar off the domain and re scan. If the crawl is fine maybe I will move the calendar to a separate domain permanently.
-
Hi.
https://publib.boulder.ibm.com/infocenter/discover/v8r4/index.jsp?topic=/com.ibm.discovery.es.ad.doc/monitoring/iiysawhttp.htm says it's an error that occurred when the crawler attempted to connect to your Web server. It says a slow site or network might be the cause of the problem.
Your robots.txt is set correctly and your site is accessible (I just tried), perhaps your best bet is to wait and see if the problem recurs.
I also ran a scan of your site using screaming frog and got close to 2,000 internal server errors (response code 500) primarily from pages in this directory - http://mississaugakids.com/mississauga-events-calendar/. The pages are loading very slowly. That might be contributing to your problem.
Maybe start there and then circle back to see if the 612 error is recurring? I'm not very technical, but perhaps they're somehow related?
-
Yeah, your robot.txt seems fine, but the answer sounded like the error code could be misleading, so maybe you're looking in the wrong area for the root of the problem due to that reason. Wish I could be of more help.
-
Hello William, Thnaks for the heads up on that thread. I did see it, however my robot.txt file should be correct, so the answer in that thread did not help in my case. Looking for other options that could be the problem here. Cheers !
-
This was brought up a little while ago, hopefully Chiaryn's answer here can help: http://moz.com/community/q/without-robots-txt-no-crawling
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Expired domain 404 crawl error
I recently purchased a Expired domain from auction and after I started my new site on it, I am noticing 500+ "not found" errors in Google Webmaster Tools, which are generating from the previous owner's contents.Should I use a redirection plugin to redirect those non-exist posts to any new post(s) of my site? or I should use a 301 redirect? or I should leave them just as it is without taking further action? Please advise.
Technical SEO | | Taswirh1 -
From page 1th to page 18th @ Google
Hello Mozzers! I have a question, you may help.. How may it be possible that a page ranking well (1th result) goes from 1th result to the 18th page just in 1 day? It doesnt seem to be any kind of penalization.. I now had all suspicious outgoing links to be nofollow (they were not before), this may be a cause .. (?) Do you have any other suggestion? Thanks
Technical SEO | | socialengaged0 -
Crawl Diagnostics and Duplicate Page Title
SOMOZ crawl our web site and say we have no duplicate page title but Google Webmaster Tool says we have 641 duplicate page titles, Which one is right?
Technical SEO | | iskq0 -
How can I see the SEO of a URL? I need to know the progress of a specific landing-page of my web. Not a keyword, an url please. Thanks.
I need to know the evolution on SEO of a specific landing-page (an URL) of my web. Not a keyword, a url. Thanks. (Necesito saber si es posible averiguar el progreso de una URL específica en el posicionamiento de Google. Es decir, lo que hace SEOmoz con las palabras clave pero al revés. Yo tengo una url concreta que quiero posicionar en las primeras posiciones de Google pero quiero ver cómo va progresando en función a los cambios que le voy aplicando. Muchas gracias)
Technical SEO | | online_admiral0 -
Can I optimize two different pages with very similar keywords without hurting SEO?
Hi there, I have often heard that you cannot have multiple pages rank for the same keyword. My question here is more about long tail keywords who have the same keyword phrase repeated on different pages. For Example: I have two webpages with different content. I want to have one page (Homepage) rank for the more generic term such as "innovation management" and another supporting page rank for "innovation management software". Will Google see these two different webpages as competing? Should I avoid repeating the more general term in the phrase? Has anyone ever seen your SEO results decline when doing this? I don't believe this is duplicate content since the pages hold completely different copy and assets but I am not sure if the repeating phrase in the title tags will flag anything to the search engines.
Technical SEO | | Scratch_MM0 -
132 pages reported as having Duplicate Page Content but I'm not sure where to go to fix the problems?
I am seeing “Duplicate Page Content” coming up in our
Technical SEO | | danatanseo
reports on SEOMOZ.org Here’s an example: http://www.ccisolutions.com/StoreFront/product/williams-sound-ppa-r35-e http://www.ccisolutions.com/StoreFront/product/aphex-230-master-voice-channel-processor http://www.ccisolutions.com/StoreFront/product/AT-AE4100.prod These three pages are for completely unrelated products.
They are returning “200” status codes, but are being identified as having
duplicate page content. It appears these are all going to the home page, but it’s
an odd version of the home page because there’s no title. I would understand if these pages 301-redirected to the home page if they were obsolete products, but it's not a 301-redirect. The referring page is
listed as: http://www.ccisolutions.com/StoreFront/category/cd-duplicators None of the 3 links in question appear anywhere on that page. It's puzzling. We have 132 of these. Can anyone help me figure out
why this is happening and how best to fix it? Thanks!0 -
Indexed pages and current pages - Big difference?
Our website shows ~22k pages in the sitemap but ~56k are showing indexed on Google through the "site:" command. Firstly, how much attention should we paying to the discrepancy? If we should be worried what's the best way to find the cause of the difference? The domain canonical is set so can't really figure out if we've got a problem or not?
Technical SEO | | Nathan.Smith0 -
Is it better to delete web pages that I don't want anymore or should I 301 redirect all of the pages I delete to the homepage or another live page?
Is it better for SEO to delete web pages that I don't want anymore or should I 301 redirect all of the pages I delete to the homepage or another live page?
Technical SEO | | CustomOnlineMarketing0