Rogerbot not showing in logs
-
Hi All
Rogerbot has recently thrown up 403 errors for all our pages - no changes had been made to the site so I asked our ISP for assistance.
They wanted to have a look at what rogerbot was doing and so went to the logs but rogerbot was not listed anywhere in the logs by name - any ideas why?
Regards
Craig
-
Hey Craig,
Sorry about the issues you are experiencing with the crawler. I just opened up a customer support ticket for you so we can investigate this on our end to see what's going wrong.
My apologies for the inconvenience!
Kenny
-
Have you performed a crawl test? That might give you a clearer idea of what's going on.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why Only Our Homepage Can Be Crawled Showing a Redirect Message as the Meta Title
Hello Everyone, So recently when we checked our domain using a Moz Crawl Test and Screaming Frog only the homepage comes up and the meta title says “You are being redirected to…”. We have several pages that used to come up and when submitting them to GSC no issues come up. The robots.txt file looks fine as well. We thought this might be ‘server’ related but it’s a little out of our field of expertise so we thought we would find out if anyone has any experience with this (ideas of reasons, how to check…etc.) or any potential suggestions. Any extra insight would be really appreciated. Please let us know if there is anything we could provide further details for that might. Looking forward to hearing from all of you! Thanks in advance. Best,
Moz Pro | | Ben-R0 -
Problem to log into moz
Every time the moz logs me out from the account and then I can not log in. It shows on the left side my name like I am logged in and then when I want go to community suddenly I am not logged in. It offen shows 502 error. It was first doing on firefox, then I manage to log in chrome and now I had to log in private browsing.
Moz Pro | | Rebeca12 -
Is there a report that shows how many times your keywords are searched?
I see the keyword ranking report but I need a report that shows the number of searches on those keywords, Thanks!
Moz Pro | | AFSB0 -
Rogerbot does not catch all existing 4XX Errors
Hi I experienced that Rogerbot after a new Crawl presents me new 4XX Errors, so why doesn't he tell me all at once? I have a small static site and had 9 crawls ago 10 4XX Errors, so I tried to fix them all.
Moz Pro | | inlinear
The next crawl Rogerbot fount still 5 Errors so I thought that I did not fix them all... but this happened now many times so that I checked before the latest crawl if I really fixed all the errors 101%. Today, although I really corrected 5 Errors, Rogerbot digs out 2 "new" Errors. So does Rogerbot not catch all the errors that have been on my site many weeks before? Pls see the screenshot how I was chasing the errors 😉 404.png0 -
Report is showing 2 different domains?
Hello SEOMOZ Fam, I am trying to get a understanding on what is the best method for me to do moving forward. I am just noticing that my reports are showing data for both http://domain & http://www.domain which is what has been confusing me for this whole past month. I have just went into my control panel and redirected domains to be just http://domain....will me doing this redirect clean up my next crawl report next week or are there other steps I need to do with my domain or even my campaign? I think I chose root level for my campaign setting. Thanks so much n advance
Moz Pro | | 41global0 -
Www. part of url not showing on google search results.
When typing in fun translator into google UK, my website www.funtranslator.com is the 18th result on the 2nd page, however, there is an issue, it only shows funtranslator.com, without the www part, which is not what I want because the whole address to show up, because my campaign on seomoz pro and webmaster tools account is www.funtranslator.com not funtranslator.com. How can I resolve this. For not, I have a redirect that goes from funtranslator to www.funtranslator.com I want to know how to get the full www.url into the results instead of just url.com without the www part of the url. Also, will this affect my stats gathered in my campaign on my seomoz pro account?
Moz Pro | | RyanSMurphy1 -
Site is showing forwarded /301 to another website
My site http://riyas.in is showing a 301 redirect or a forward to http://flicker.com/muhammedriyas . I had done a 301 redirect long before from my site to this domain, but i removed that after 2-3 days. Please help me to solve this problem. I attached a screen shot seomoz1.jpg
Moz Pro | | riyas_0 -
Is there a whitelist of the RogerBot IP Addresses?
I'm all for letting Roger crawl my site, but it's not uncommon for malicious spiders to spoof the User-Agent string. Having a whitelist of Roger's IP addresses would be immensely useful!
Moz Pro | | EricCholis1