undefined
Skip to content
Moz logo Menu open Menu close
  • Products
    • Moz Pro
    • Moz Pro Home
    • Moz Local
    • Moz Local Home
    • STAT
    • Moz API
    • Moz API Home
    • Compare SEO Products
    • Moz Data
  • Free SEO Tools
    • Domain Analysis
    • Keyword Explorer
    • Link Explorer
    • Competitive Research
    • MozBar
    • More Free SEO Tools
  • Learn SEO
    • Beginner's Guide to SEO
    • SEO Learning Center
    • Moz Academy
    • SEO Q&A
    • Webinars, Whitepapers, & Guides
  • Blog
  • Why Moz
    • Agency Solutions
    • Enterprise Solutions
    • Small Business Solutions
    • Case Studies
    • The Moz Story
    • New Releases
  • Log in
  • Log out
  • Products
    • Moz Pro

      Your all-in-one suite of SEO essentials.

    • Moz Local

      Raise your local SEO visibility with complete local SEO management.

    • STAT

      SERP tracking and analytics for enterprise SEO experts.

    • Moz API

      Power your SEO with our index of over 44 trillion links.

    • Compare SEO Products

      See which Moz SEO solution best meets your business needs.

    • Moz Data

      Power your SEO strategy & AI models with custom data solutions.

    NEW Keyword Suggestions by Topic
    Moz Pro

    NEW Keyword Suggestions by Topic

    Learn more
  • Free SEO Tools
    • Domain Analysis

      Get top competitive SEO metrics like DA, top pages and more.

    • Keyword Explorer

      Find traffic-driving keywords with our 1.25 billion+ keyword index.

    • Link Explorer

      Explore over 40 trillion links for powerful backlink data.

    • Competitive Research

      Uncover valuable insights on your organic search competitors.

    • MozBar

      See top SEO metrics for free as you browse the web.

    • More Free SEO Tools

      Explore all the free SEO tools Moz has to offer.

    NEW Keyword Suggestions by Topic
    Moz Pro

    NEW Keyword Suggestions by Topic

    Learn more
  • Learn SEO
    • Beginner's Guide to SEO

      The #1 most popular introduction to SEO, trusted by millions.

    • SEO Learning Center

      Broaden your knowledge with SEO resources for all skill levels.

    • On-Demand Webinars

      Learn modern SEO best practices from industry experts.

    • How-To Guides

      Step-by-step guides to search success from the authority on SEO.

    • Moz Academy

      Upskill and get certified with on-demand courses & certifications.

    • SEO Q&A

      Insights & discussions from an SEO community of 500,000+.

    Unlock flexible pricing & new endpoints
    Moz API

    Unlock flexible pricing & new endpoints

    Find your plan
  • Blog
  • Why Moz
    • Small Business Solutions

      Uncover insights to make smarter marketing decisions in less time.

    • Agency Solutions

      Earn & keep valuable clients with unparalleled data & insights.

    • Enterprise Solutions

      Gain a competitive edge in the ever-changing world of search.

    • The Moz Story

      Moz was the first & remains the most trusted SEO company.

    • Case Studies

      Explore how Moz drives ROI with a proven track record of success.

    • New Releases

      Get the scoop on the latest and greatest from Moz.

    Surface actionable competitive intel
    New Feature

    Surface actionable competitive intel

    Learn More
  • Log in
    • Moz Pro
    • Moz Local
    • Moz Local Dashboard
    • Moz API
    • Moz API Dashboard
    • Moz Academy
  • Avatar
    • Moz Home
    • Notifications
    • Account & Billing
    • Manage Users
    • Community Profile
    • My Q&A
    • My Videos
    • Log Out

The Moz Q&A Forum

  • Forum
  • Questions
  • Users
  • Ask the Community

Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

  1. Home
  2. Moz Tools
  3. Replicate rogerbot error for server/hosting provider

Moz Q&A is closed.

After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.

Unsolved Replicate rogerbot error for server/hosting provider

Moz Tools
robots roger-bot
16
22
3.8k
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as question
Log in to reply
This topic has been deleted. Only users with question management privileges can see it.
  • Moving-Web-SEO-Auckland
    Moving-Web-SEO-Auckland Subscriber last edited by May 4, 2022, 3:19 AM

    Anyone got any ideas how to get a server/hosting provider who is preventing rogerbot from crawling and me not been able to set up a campaign to duplicate the error on there end?

    The server/hosting provider is crazydomains dot com

    My clients robots.txt

    User-agent: *
    Disallow: /wp-admin/
    Allow: /wp-admin/admin-ajax.php
    User-agent: rogerbot

    Disallow:

    Sitemap: https://www. something

    Tom.Capper sam0090 2 Replies Last reply Jun 25, 2024, 4:42 AM Reply Quote 0
    • JuliaRobets891
      JuliaRobets891 last edited by Sep 23, 2024, 5:50 PM

      thanks guys for this post

      1 Reply Last reply Reply Quote 0
      • topic:timeago_earlier,24 days
      • alexseen111
        alexseen111 last edited by Aug 30, 2024, 4:15 PM

        A friend of mine told me about online gaming, and I got curious. I never tried these kinds of sites before, so I figured I’d check it out. I wanted something that wasn’t too complicated, and that’s when I came across 1win-game.co.in. The site looked cool and was easy to navigate, so I gave it a shot. They’ve got a bunch of cool offers like sign-up bonuses and special promotions, which made me feel like I was getting some extra value right off the bat. I explored a few games, and the whole experience was smooth and enjoyable. The bonuses were easy to understand and use, which was perfect for a beginner like me. What really stood out was how everything was straightforward and hassle-free. I’m glad I found this site.

        1 Reply Last reply Reply Quote 0
        • alexseen111
          alexseen111 last edited by Aug 30, 2024, 4:13 PM

          вфівіфвіфвфівфів

          1 Reply Last reply Reply Quote 0
          • weray609
            weray609 last edited by Aug 26, 2024, 6:21 PM

            hello i like this post !

            1 Reply Last reply Reply Quote 0
            • topic:timeago_earlier,20 days
            • john1432
              john1432 last edited by Aug 6, 2024, 9:20 AM

              his thing is great thanks for help

              the information provided,
              they helped me a lot Edu connect. I hope you have more information Great blog.

              1 Reply Last reply Reply Quote 0
              • john1432
                john1432 last edited by Aug 6, 2024, 7:37 AM

                I'm glad you found the information helpful on. Thanks for the information provided,
                they helped me a lot trust I hope you have more information Great blog.

                1 Reply Last reply Reply Quote 0
                • iueowe
                  iueowe @sam0090 last edited by Aug 6, 2024, 4:49 AM

                  @sam0090 I am a student myself and don't know much about Math but after doing a lot of research I came across a reliable website called calculadora-de-horas(https://calculadora-de-horas.com/como-calcular-44-horas-de-trabalho-de-segunda-a-sexta-feira/) from here I got information about calculators. I. In this, I learned about the difficult calculation, and here is a noteworthy point I had such problems with my site too, I corrected it after seeing your reply. thanks

                  1 Reply Last reply Reply Quote 0
                  • teoin
                    teoin @sam0090 last edited by Aug 6, 2024, 4:23 AM

                    @sam0090 You have helped a lot by expressing your thoughts because this problem was with my website which was related to the calculator. I found a solution to my problem by following your entire method and preparing my website to work correctly and better. Thank you.

                    1 Reply Last reply Reply Quote 0
                    • Mike247
                      Mike247 last edited by Aug 5, 2024, 8:52 PM

                      Webugol approached our project with a strategic mindset, ensuring that every aspect of the website aligned with our business goals. Their effective planning and execution have resulted in a site that not only looks great but also drives our business forward. We are extremely satisfied with the results and grateful for their hard work. Let's check this site https://webugol.com/blog/innovative-marketing-ideas-for-chiropractors-to-stand-out

                      1 Reply Last reply Reply Quote 0
                      • topic:timeago_earlier,14 days
                      • sam0090
                        sam0090 @sam0090 last edited by Jul 23, 2024, 4:36 AM

                        @sam0090 Certainly! The issue with Cookiebot, GTM, and Typo3 seems to be related to script blocking. Make sure essential scripts for your hover menu are correctly categorized in Cookiebot’s settings to prevent them from being blocked. Also, review GTM’s integration to ensure it adheres to Cookiebot's consent settings.

                        teoin 1 Reply Last reply Aug 6, 2024, 4:23 AM Reply Quote 0
                        • shinobi1193
                          shinobi1193 @monkaw last edited by Jul 22, 2024, 2:57 AM

                          @monkaw said in Replicate rogerbot error for server/hosting provider:

                          I've been fond of football since I was a kid, and recently I decided to place online bets on football matches on this site - https://betvisa.guru/Slope Game - and as it turned out, it's a great way to win money in Bangladesh! I recommend this site to you too if you love sports and want to place online bets on them.

                          Thank monkaw. This is great advice and you are very generous in sharing it.

                          1 Reply Last reply Reply Quote 0
                          • Roberto99881
                            Roberto99881 last edited by Jul 21, 2024, 9:13 AM

                            When my AC started acting up, I was unsure how to handle it. I found https://serviceservotech.com/air-conditioning/ac-repair-installation/ and was impressed by their comprehensive advice on both repair and installation. The information was clear and easy to follow, allowing me to address the issues effectively. If you need reliable guidance for your air conditioning system, this site is a great resource.

                            1 Reply Last reply Reply Quote 0
                            • topic:timeago_earlier,24 days
                            • JuliaRobets891
                              JuliaRobets891 last edited by Jun 27, 2024, 3:25 AM

                              Trying to find ways to break through my muscle growth plateau, I came across this insightful piece on retatrutide. The article’s focus on how it enhances muscle growth and recovery was particularly helpful. It provided clear, actionable information. If you're in a similar situation, I recommend visiting https://valhallavitality.com/blog/the-revolutionary-influence-of-retatrutide-on-bodybuilding-maximizing-muscle-growth-and-recovery for a detailed guide on how this supplement can support your bodybuilding goals.

                              1 Reply Last reply Reply Quote 0
                              • sam0090
                                sam0090 @Moving-Web-SEO-Auckland last edited by Jun 25, 2024, 4:42 AM

                                @Moving-Web-SEO-Auckland It sounds like you're having trouble with rogerbot being blocked by your hosting provider, CrazyDomains. Here are a few steps you can take:

                                Check Robots.txt Syntax: Ensure your robots.txt file is correctly formatted. It seems mostly correct, but it should be double-checked for any syntax errors.

                                makefile
                                Copy code
                                User-agent: *
                                Disallow: /wp-admin/
                                Allow: /wp-admin/admin-ajax.php

                                User-agent: rogerbot
                                Disallow:

                                Sitemap: https://www.something.com/sitemap.xml
                                Contact Hosting Support: Reach out to CrazyDomains support and explain the issue. They may be blocking rogerbot on the server side. Provide them with specific details about your issue and ask if there are any server-side restrictions for rogerbot.

                                Use Alternative Bots: If the issue persists, you could test with other bots to see if the same restrictions apply. This can help identify if the problem is specific to rogerbot or a broader issue.

                                Server Logs: Check your server logs to see if there are any entries that show why rogerbot is being blocked. This can provide more insight into what’s happening.

                                Campaign Setup: To duplicate the error, you might set up a staging environment with the same robots.txt and see if the issue persists there. This can help you troubleshoot without affecting the live site.

                                If these steps don't resolve the issue, you may need to consider switching to a different hosting provider that allows more flexibility with bot access.

                                sam0090 iueowe 2 Replies Last reply Aug 6, 2024, 4:49 AM Reply Quote 0
                                • JuliaRobets891
                                  JuliaRobets891 last edited by Jun 25, 2024, 3:31 AM

                                  I've been exploring the idea of hormone therapy to boost my wellness journey. Initially, I was a bit skeptical, but after reading through a resource I found https://ways2well.com/blog/whats-wrong-with-health-insurance, I'm considering giving it a go. It claims to offer impressive benefits such as anti-aging and immune system enhancement. I'd appreciate if anyone here has had any personal experience with this to share their thoughts.

                                  1 Reply Last reply Reply Quote 0
                                  • topic:timeago_earlier,7 days
                                  • monkaw
                                    monkaw last edited by Jun 17, 2024, 3:58 PM

                                    I've been fond of football since I was a kid, and recently I decided to place online bets on football matches on this site - https://betvisa.guru/ - and as it turned out, it's a great way to win money in Bangladesh! I recommend this site to you too if you love sports and want to place online bets on them.

                                    shinobi1193 1 Reply Last reply Jul 22, 2024, 2:57 AM Reply Quote 0
                                    • topic:timeago_earlier,18 days
                                    • SandraHall
                                      SandraHall last edited by May 30, 2024, 9:26 PM

                                      Thank you all, thanks to you, I solved my problem and learned a lot of new information for myself

                                      1 Reply Last reply Reply Quote 0
                                      • topic:timeago_earlier,2 months
                                      • Faraon23
                                        Faraon23 last edited by Mar 23, 2024, 4:45 PM

                                        Hi guys. I decided to try this site https://nocramming.com/testingmom-review to prepare my child for testing before entering a new school, and the results have exceeded my expectations. TestingMom provides an extensive selection of materials and exercises to help children improve their academic skills and develop problem-solving strategies. My child has improved quickly with this resource and I am very grateful to the TestingMom team for their work

                                        1 Reply Last reply Reply Quote 0
                                        • topic:timeago_earlier,6 months
                                        • Leasedlayer
                                          Leasedlayer last edited by Sep 21, 2023, 12:50 PM

                                          Moving-Web-SEO-AucklandMoving-Web-SEO-Auckland

                                          if you facing any error in your hosting kindly contact the https://www.leasedlayer.com/ I would like to recommend this company because I have hosted my company site with this service provider I am running my site in a smooth way and their customer support is marvelous

                                          1 Reply Last reply Reply Quote 0
                                          • topic:timeago_earlier,about a year
                                          • Moving-Web-SEO-Auckland
                                            Moving-Web-SEO-Auckland Subscriber @Tom.Capper last edited by May 4, 2022, 11:20 PM

                                            @tom-capper
                                            Thanks Tom was more about server settings outside of robots.txt for hosting providers.
                                            Managed to get them to remove rogerbot blocking on ALL there servers as they first stated that it was well known as a malicious robot hence the block???
                                            Until I made them aware.... cough cough

                                            1 Reply Last reply Reply Quote 0
                                            • Tom.Capper
                                              Tom.Capper Subject Expert @Moving-Web-SEO-Auckland last edited by May 4, 2022, 1:46 PM

                                              @moving-web-seo-auckland

                                              There are a bunch of third party tools that would allow you to validate robots.txt against a given user agent.

                                              For example:

                                              • https://technicalseo.com/tools/robots-txt/ (includes Moz user agents in dropdown)
                                              • https://www.realrobotstxt.com/ (allows you to copy/paste a test robots file)

                                              Without the hosting provider having a Moz account & campaign of their own, this is probably the easiest way to get them to replicate the error.

                                              Tom

                                              Moving-Web-SEO-Auckland 1 Reply Last reply May 4, 2022, 11:20 PM Reply Quote 0
                                              • 1 / 1
                                              1 out of 22
                                              • First post
                                                1/22
                                                Last post

                                              Got a burning SEO question?

                                              Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.


                                              Start my free trial


                                              Browse Questions

                                              Explore more categories

                                              • Moz Tools

                                                Chat with the community about the Moz tools.

                                              • SEO Tactics

                                                Discuss the SEO process with fellow marketers

                                              • Community

                                                Discuss industry events, jobs, and news!

                                              • Digital Marketing

                                                Chat about tactics outside of SEO

                                              • Research & Trends

                                                Dive into research and trends in the search industry.

                                              • Support

                                                Connect on product support and feature requests.

                                              • See all categories

                                              Related Questions

                                              • Skites2

                                                Unsolved 403 errors for assets which work fine

                                                moz pro 403 errors backlinkaudit

                                                Hi,
                                                I am facing some issue with our moz pro account
                                                We have images stored in a s3 buckets eg: https://assets2.hangrr.com/v7/s3/product/151/beige-derby-cotton-suit-mb-2.jpg
                                                Hundreds of such images show up in link opportunities - Top pages tool - As 403 ... But all these images work fine and show status 200. Can't seem to solve this. Thanks.

                                                Moz Tools | Dec 10, 2024, 4:07 PM | Skites2
                                                0
                                              • seotoolsking

                                                Unsolved How to check the Domain Age?

                                                seo tools seo audit

                                                Check the Domain Age on THE SEO TOOLS KING
                                                Get Free seo tools only on THE SEO TOOLS KING

                                                Moz Tools | Dec 12, 2024, 11:34 AM | seotoolsking
                                                5
                                              • BB_NPG

                                                Unsolved Rogerbot blocked by cloudflare and not display full user agent string.

                                                roger-bot crawl error

                                                Hi, We're trying to get MOZ to crawl our site, but when we Create Your Campaign we get the error:
                                                Ooops. Our crawlers are unable to access that URL - please check to make sure it is correct. If the issue persists, check out this article for further help. robot.txt is fine and we actually see cloudflare is blocking it with block fight mode. We've added in some rules to allow rogerbot but these seem to be getting ignored. If we use a robot.txt test tool (https://technicalseo.com/tools/robots-txt/) with rogerbot as the user agent this get through fine and we can see our rule has allowed it. When viewing the cloudflare activity log (attached) it seems the Create Your Campaign is trying to crawl the site with the user agent as simply set as rogerbot 1.2 but the robot.txt testing tool uses the full user agent string rogerbot/1.0 (http://moz.com/help/pro/what-is-rogerbot-, rogerbot-crawler+shiny@moz.com) albeit it's version 1.0. So seems as if cloudflare doesn't like the simple user agent. So is it correct the when MOZ is trying to crawl the site it uses the simple string of just rogerbot 1.2 now ? Thanks
                                                Ben Cloudflare activity log, showing differences in user agent strings
                                                2022-07-01_13-05-59.png

                                                Moz Pro | Jul 1, 2022, 12:14 PM | BB_NPG
                                                0
                                              • amandacash85896

                                                Unsolved Is Performance Metrics only available in a Campaign?

                                                crawl performance

                                                I'm looking to do a 1-off Performance Metrics analysis across dozens of pages on a single website - a prospective client. I thought it would be part of the On-Demand Crawl.

                                                Moz Tools | Dec 11, 2024, 1:57 PM | amandacash85896
                                                0
                                              • serktba

                                                Unsolved reduce spamscore

                                                spam score white hat

                                                Hello
                                                How can I reduce the spamscore on my website?
                                                All the activities I did on my website were completely white hat and I did not do any spam jobs, but my spam score is 2%
                                                This is my site address
                                                https://abedinigroup.com/
                                                Please help me how can I reduce the spamscore in my account?

                                                Moz Tools | Dec 12, 2024, 3:16 PM | serktba
                                                9
                                              • peaceforchange00

                                                Unsolved Domain Authority Not Changed?

                                                moz

                                                The domain authority has not changed for a few months; even I got some high-quality links from authority sites, and Moz has not crawled them yet? Can someone tell me how much time it generally Moz takes to crawl new links for a domain and update DA?
                                                My domain is https://thebookspdf.net/

                                                Moz Tools | Dec 11, 2024, 7:18 PM | peaceforchange00
                                                10
                                              • semrtaz

                                                Unsolved domain authority has not changed

                                                domain authority link tracking list link explorer backlinks

                                                Hello, can you help me? The domain authority has not changed for months, and in fact, the ranking has come down. But my links have increased a lot but have remained the same
                                                this is my domain 😧 https://nezafatnews.ir/

                                                Moz Tools | Dec 12, 2024, 11:57 AM | semrtaz
                                                7
                                              • valigholami1386

                                                Unsolved error in crawling

                                                crawl crawl errors

                                                hello moz . my site is papion shopping but when i start to add it an error appears that it cant gather any data in moz!! what can i do>???

                                                Moz Tools | May 21, 2024, 8:40 AM | valigholami1386
                                                0

                                              Get started with Moz Pro!

                                              Unlock the power of advanced SEO tools and data-driven insights.

                                              Start my free trial
                                              Products
                                              • Moz Pro
                                              • Moz Local
                                              • Moz API
                                              • Moz Data
                                              • STAT
                                              • Product Updates
                                              Moz Solutions
                                              • SMB Solutions
                                              • Agency Solutions
                                              • Enterprise Solutions
                                              Free SEO Tools
                                              • Domain Authority Checker
                                              • Link Explorer
                                              • Keyword Explorer
                                              • Competitive Research
                                              • Brand Authority Checker
                                              • Local Citation Checker
                                              • MozBar Extension
                                              • MozCast
                                              Resources
                                              • Blog
                                              • SEO Learning Center
                                              • Help Hub
                                              • Beginner's Guide to SEO
                                              • How-to Guides
                                              • Moz Academy
                                              • API Docs
                                              About Moz
                                              • About
                                              • Team
                                              • Careers
                                              • Contact
                                              Why Moz
                                              • Case Studies
                                              • Testimonials
                                              Get Involved
                                              • Become an Affiliate
                                              • MozCon
                                              • Webinars
                                              • Practical Marketer Series
                                              • MozPod
                                              Connect with us

                                              Contact the Help team

                                              Join our newsletter
                                              Moz logo
                                              © 2021 - 2025 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
                                              • Accessibility
                                              • Terms of Use
                                              • Privacy

                                              Looks like your connection to Moz was lost, please wait while we try to reconnect.