Need to solve "Oops our crawlers were unable to access" url for new campaign
-
I'm putting the url designfirstkitchenandbath.com and getting the "oops! our crawlers were unable to access the site. Since this site is a potential client, which shows up online, I can't get access to fix the code, plus while I can write a little html I don't feel comfortable working with hard, live code on someonelse's site. Anyone have a simple solution?
-
Hey there!
Sam from Moz's Help Team here!
I've taken a look at that URL, and from what I can see it looks like we're running into a 403 error which means that ultimately, access to that URL is forbidden:
http://www.screencast.com/t/UbMwIrjuQFG
This 403 error is coming directly from your server, so at this point the best thing to do will be to send this information to your client/their web dev. They'll need to make sure our crawler (rogerbot) is whitelisted, so our tools can start working hard for you, and that you aren't blocking AWS (Amazon Web Services) on the server level.
I hope this helps - if you have any other questions, feel free to pop a message over to help@moz.com
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
New Domain Authority 2.0 have affected my website rankings badly
My website https://www.successvalley.tech/ domain authority was 39 but after the new update, its now 15. Am not happy at all because it took me three years to get that and now it has been reduce to nothing. Please I kindly need explanation as to what happened. Thanks
Moz Bar | | Amenorhu1 -
MOZ crawler 404 errors on wordpress
Hi all, I've got hundreds of issues coming up on the MOZ crawler with 404 errors, I don't know what these URL's are. Here's a couple of examples; http://www.theswagbagco.co.uk/category/watford/http%3A%2F%2Fwww.theswagbagco.co.uk%2F2015%2F10%2F15%2Fnew-products-2%2F
Moz Bar | | vaineh
http://www.theswagbagco.co.uk/2015/10/01/thank-you-epsom/http%3A%2F%2Fwww.theswagbagco.co.uk%2F2015%2F10%2F01%2Fthank-you-epsom%2F See the first one is one page with a different url appended, the second is the same thank-you-epsom url. How would I find out where these are even being linked from?0 -
URL Link Counts
Hi Can someone clarify what the URL link counts in the SERP report means? Is this total internal links on this page? My followed internal links are very high compared with sites I am comparing against, some sites only have 1 internal followed link - is this possible? Have they simply no followed all the rest on that page?
Moz Bar | | BeckyKey0 -
Cannot crawl website with redirect intalled on subdomain url
Hi! I want to crawl this website : http://www.car-moderne.ch. I tried a got back the crawl just for that one url (not for all the pages of the website). This single line cvs says that the status of the http://www.car-moderne.ch is 200, but in fact it is a redirect 301 to http://www.car-moderne.ch/fr where the live home page is (actually the Moz bar sees the 301, not the 200 as the single-lined crawl does). How can I proceed in this case (a 301 redirect being installed on the subdomain url) to still be able to have a full-fledged juicy cvs with all the broken links, duplicate content, etc. Thank you for your help! Pascal Hämmerli
Moz Bar | | Ethos_Digital0 -
Idea for new feature on Moz Q&A
Hi Moz Can we please have a new feature to moz when answering questions please, the ability to save as a draft. I am currently on the train from London back home and I was trying to answer a few questions and give my opinion and after writing a really long and detailed response to an area I actually have quite a lot to say on, my internet froze and the page refreshed losing all the content. It doesn’t help that my Mac I am working on isn’t the fastest, but my new one isn’t quite ready yet. But if there was a simple save draft feature, it would be really great as I wasn’t ready to post my answer as it wasn’t finished and without checking wouldn’t have made sense. Yes I know I could of done it in word and pasted it across, but usually when I am travelling and answering questions I do it on my Ipad which isn’t great for multitasking. While I understand you don’t want lots of ‘draft answers’ all over the place as this would take storage space up on your servers and doesn’t actually help the person answering the question, your system could purge all draft answers which have not been edited in the last 24 hours. Just a little suggestion to help make this great forum even better. Thanks Andy
Moz Bar | | Andy-Halliday4 -
I'm getting, "you're not using the rel="canonical" META attribute" in my crawl diagnotic
I'm running a campaign crawler through Moz on this particular page: http://www.henley.ac.uk/executive-education/leadership-and-management-programmes/ but I'm getting a notifcaiton from Moz saying, "you're not using the rel="canonical" META attribute" I don't understand what this means!! Has anyone else had this problem, or can they help me understand what this means and how to fix it? Oh, and Happy Thanksgiving from the UK! Virginia
Moz Bar | | blackboxideas0 -
Moz Crawler not Identifying all Duplicate Pages
On two recent site crawls (9/27/14 and 11/4/14) for duplicate content the Moz tool did not ID the following 2 pages, which are 100% duplicate to each other: http://www.hooksandlattice.com/planter-hampton-241212.html ; Screenshot: http://screencast.com/t/DdwWroUU http://www.hooksandlattice.com/planter-hampton-721212.html ; Screenshot: http://screencast.com/t/8Lb1cJZmGrhX As I'm working feverishly to re-write and update the site (goal is ZERO duplicates) I'm finding it challenging to use the Moz tool to get the project done. Does anyone have any feedback or help they can provide for how I can identify all duplicate pages associated with my domain? Thank you! Lindsey Pfeiffer
Moz Bar | | CMC-SD0 -
Moz Local | Empty page "Categories"
Dear Moz, Another error, the following url loads an empty page https://moz.com/local/categories Please review Thanks!
Moz Bar | | Bio-RadAbs0