Can't Crawl Site - but deducting crawls.
-
Why am I being deducted crawls if MOZ keeps telling me that it can't crawl my site?
-
Any time you initiate a crawl, it counts against your limits, even if something on the site is blocking us from crawling it. I would suggest that write into help@moz.com so we can help you diagnose what is blocking us from crawling your site.
-
Sometimes, when running a crawl on a site (or a section of a site), you may find that it isn’t progressing past the first level of URLs. When this happens, only the base domain, or “start URLs” are actually crawled
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved What would the exact text be for robots.txt to stop Moz crawling a subdomain?
I need Moz to stop crawling a subdomain of my site, and am just checking what the exact text should be in the file to do this. I assume it would be: User-agent: Moz
Getting Started | | Simon-Plan
Disallow: / But just checking so I can tell the agency who will apply it, to avoid paying for their time with the incorrect text! Many thanks.0 -
Changing the Moz Crawl Date
Hello, I am wondering where I can change the date of Crawl by Moz. I would like to change this crawl period from one week to 2 or even 3 weeks for Moz to crawl my website. Hope to hear from anyone soon. Kind regards, Koen.
Getting Started | | Koenniiee1 -
SSL - green padlock but Moz say there's an 804 error?
Hi, my site has a green padlock and no SSL errors but Moz are reporting an 804 error. I use CloudFlare with fairly complex settings. I've read this thread but it's quite old and I don't understand which parts of it are still valid. I'd love to know whether this can be sorted before I spend hours setting up Moz's features as if they can't crawl my site then I would obviously need to cancel my subscription. Thanks
Getting Started | | Barn2Plugins0 -
New to MOZ, can't create a campaign.
I just started the free trial today, but I can't setup a campaign. Everywhere I go (http://analytics.moz.com/pro/home, http://analytics.moz.com/manage-campaigns), all I see is: Oops! Try refreshing the page, if that doesn't work, please click here to contact our help team. Is something broken?
Getting Started | | jcsilkey0 -
Is analytics data kept after PRO account is discontinued? And/or can we step back up later again, and acces it?
When we discontinue our PRO account, will we be able to step back up with the same account later? If so, is there any data of the intermediate months? And can we still access old data from during our PRO account from the free account?
Getting Started | | meamedica0 -
How do I interpret Duplicate Content in a Crawl Report, when it only gives me a URL? How do I know what is duplicated on that page somewhere else?
I need help interpreting the Crawl Report for Duplicate Content. It gives me the URLs of pages that have duplicate content, but how do I know what content exactly is duplicated elsewhere? And how do I figure out where it is duplicated? Also, are there Moz Analytics articles or videos teaching you how to use each component of the analytics programs? Thanks!
Getting Started | | NancyBryan0 -
'Domain Does Not Respond To Web Requests'
Hi everyone, This seems to be a fairly common query on the Q&A section, but I haven't been able to find a solution by reading through previous threads. When I try to set up an SEOmoz campaign for spryz.co.nz, I get that ol' favourite error message: 'We have detected that the domain spryz.co.nz does not respond to web requests. Using this domain, we will be unable to crawl your site or present accurate SERP information.' The problem isn't being caused by a Robots.txt file, and the site has experienced 99% uptime since it was launched.Traffic stats show that visits are coming through to the site via the search engines, suggesting that it may not be all crawlers that fail to access the site. I've tried to set up this campaign several times throughout the day, since I've read that sometimes Roger goes on the blink, but I've still not been successful. Any suggestions as to why Roger might be unable to crawl my website would be great.
Getting Started | | e8creative0