On Page Grader can't access my URLs
-
HI-
I am trying to grade some specific pages for keywords with the on page grader but it keeps telling me "Sorry, but that URL is inaccessible. "
I can reach them via the browser and they are not https. Any thoughts?
Here is a sample:
www.bulkcandystore.com/kosher-candy
Any help is appreciated.
Ken
-
Hey Ken,
Thanks for following up. You can add a crawl delay for rogerbot in your robots.txt file to limit how quickly we make requests to the site. I would just recommend that you don't add a crawl delay higher than 10 because that can cause us to be unable to crawl the site in a reasonable amount of time to complete a full crawl.
I hope that helps!
Chiaryn
-
Thanks for the info. I checked with my ISP and it turns out that they automatically blocked Rogerbot because of excessive requests (this happened Jan 15). Is there a way to crawl dely between requests longer or was that maybe a one time fluke?
Thanks
Ken
-
Hey Ken,
Thanks for writing in and sorry for the confusion. I'm afraid the server for www.bulkcandystore.com/kosher-candy is returning a 403 forbidden error for our crawler, so we are not allowed to access that page with our tools. The server would respond differently to the browser vs. our tools so I would recommend contacting the webmaster for the site and white-labelling the user-agent rogerbot so that we can access the site in the future.
I hope this helps. Please let me know if you have any other questions.
Chiaryn
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
MozBar: Page Title (and other data) inconsistent with HTML
In using the MozBar Chrome extension, I have noticed that the data about a page that MozBar displays is sometimes inconsistent with the "head" tag in the HTML of the page. Am I misunderstanding where MozBar is getting the data it is displaying? Other SEO extensions like "META SEO inspector"do not display these inconsistencies. EupH5sG
Moz Bar | | clayjones0 -
How to export to csv all top pages links in link explorer?
When i press export to csv it does not offer to choose how many rows but exports to file only 500 rows. But there is more than 3000. How to export all rows? Thanks
Moz Bar | | AndrisZigurs0 -
On-Page Grader giving wrong results for missing keyword in image Alt Attribute
I'm trying to optimize for the Keyword "River City Apartments" on this page: https://3fconstruction.net/case-studies/river-city-apartments-chicago-condo-deconversion-renovation/ When I use On-Page Grader to see how I'm doing it's telling me I'm missing the Keyword in the Image Alt Attribute. However, it's clearly in there. <strong>River City Apartments Aerial</strong> What am I missing?
Moz Bar | | Drew.Friestedt0 -
On-page Grader API?
Hi guys, I have a massive list of keywords i want to optimise against a specific URL. I'm checking them manually with the MOZ on-page grader to determine if the keyword is optimised for the URL (Grade A score). I was wondering if there is a way to pull the scores automatically for each URL with the MOZ api? Or does anyone know of any tools which can automate this process? Cheers, Chris
Moz Bar | | jayoliverwright1 -
Re On-Page Grader
One of the pages I'm trying to optimise is achieving an 'A' grade, however all the ticks are black not green as I've seen on other page grade. Why is this? Help much appreciated. Thanks
Moz Bar | | seoman100 -
On Page Grader not working on the fly
On Page Grader has this lovely little re-grade page button next to the URL and term field but it's never worked on the fly for me. What's going on here? You discover something critical, fix it and you want the satisfaction of re-grading to test it and get that A but it doesn't work. I can see the changes on the site so why not??
Moz Bar | | wearehappymedia0 -
Duplicate page content
Hi guys the feedback form my campaign suggests I have to much duplicate page content. I’ve had a look at the CSV file but it doesn’t seem to be abundantly clear as to which pages on my site have the duplicate content. Can anyone tell which columns I need to refer to on the sheet, to ascertain this information. Also if the content is only slightly different, will Google still consider it to be duplicate? I look forward to hearing from you
Moz Bar | | Hardley1110 -
Moz just did my weekly "crawl" and claims my meta titles and descriptions are "missing," but they aren't. What's up with this?
I can check for my titles and descriptions in the source. They are there, always have been. what in the hell?
Moz Bar | | LifeisRosie0