Crawler unable to access pages
-
Hi crawler is unable to access site and crawl properly. Mainly for the backlink checker, it's producing no results
There is nothing in the robots.txt file blocking crawler access.
Any help is much appreciated as it's driving me crazy!
-
Hey!
Sure you can write into help@moz.com and we will get you sorted out!
-
Hi.
Can I send the url privately if possible?
Thanks
-
Hey, Dave from the Help Team here
Are you receiving this error when running your site through Open Site Explorer? Can you please provide the URL that you are checking?
thanks!
-
Is there are specific errors being reported along with the crawler message or see any numeric errors on the crawl page?
Thanks,
Don
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved "The account does not have access to that quota" - API v1 URL Metrics
Hi! On one of our servers we get the message "The account does not have access to that quota" when pulling data for v1 URL Metrics endpoint in the API.
API | | aheitzman
This only happens on one specific server, others have worked flawlessly. Any idea of what might be going on? Thanks0 -
API Access Change
I'm not sure, but I seem to remember that there was a certain number of API calls that USED to be included with my MOZ Pro level subscription and as I check it now, the API access is a separate subscription. Was there a recent change or has my memory finally started going?
API | | SWGroves0 -
Does on-page grader have an API ?
Hi, I would very much like to include the on-page grader output into my SEO tools. Is there an API for that? thanks James
API | | KMdayJob0 -
Does Moz's crawlers use _escaped_fragment_ to inspect pages on a single-page application?
I just got started, but got a 902 error code on some pages, with a message saying there might be an outage on my site. That's certainly not the case, so I'm wondering if the crawlers actually respect and use the escaped_fragment query parameter. Thanks, David.
API | | CareerDean0 -
On-Page Reports showing old urls
While taking a look at our sites on-page reports I noticed some of our keywords with very old urls that haven't existed for close to a year. How do I make sure moz's keyword ranking is finding the correct page and make sure I'm not getting graded on that keywords/urls that don't exist any more or have been 301'd to new urls? Is there a way to clean these out? My on-page reports say I have 62 reports for only a total of 34 keywords in rankings. As you can see from the image most of the urls for "tax folder" have now been 301'd to not include /product or /category but moz is still showing them with the old url structure. BTW our site is minespress.com 2KdGcPL.png
API | | smines0 -
Top Pages metrics in OSE
Not sure if this is an API question or a feature request, but wondering if other folks had a way to do this: In OSE there is the dashboard for a specific URL that is entered into the search bar, giving you metrics at a glance on it. But I often find myself going to the Top Pages tab to get a sense of the domain as a whole. First off, wouldn't it be nice/is there a way to build my own "dashboard" based on info from that section? Specifically, I'd love to see at a glance the number of "top pages" that exist (many websites are well under the 10,000 page limit for this section, but there's no quick glance metric showing that) One thing that would be very handy for me would be a breakdown of HTTP Status info across the whole domain, as being able to see the raw total of different statuses (and the percentages of each based on the total number of pages) would be really helpful, giving me a sense if I should dig into any issues before exporting the list to CSV. I've found myself needing this type of info for multiple domains at once, so what would be REALLY cool would be a Google Doc where I could paste in different domains in one column, and this info being returned in other columns. I've searched through the Q&A and didn't find anything like this, and I didn't know how easy/hard any of this would be to do, but I was wondering if anyone else had a sense of how to solve this problem and how feasible it would be to tackle it. Thanks!
API | | John-E-Turner0 -
Huge in crease in on page errors
Hi guys I’ve just checked my online campaign and I see errors in my crawl diagnostics have almost doubled from the 21<sup>st</sup> of October to the 25<sup>th</sup> of October going from 6708 errors to 11 599. Can anyone tell me what may have caused this? Also I notice we have a lot of issues with duplicate page titles which seems strange as no new pages have been added, can anyone explain why this might be? I look forward to hearing from you
API | | Hardley1110