undefined
Skip to content
Moz logo Menu open Menu close
  • Products
    • Moz Pro
    • Moz Pro Home
    • Moz Local
    • Moz Local Home
    • STAT
    • Moz API
    • Moz API Home
    • Compare SEO Products
    • Moz Data
  • Free SEO Tools
    • Domain Analysis
    • Keyword Explorer
    • Link Explorer
    • Competitive Research
    • MozBar
    • More Free SEO Tools
  • Learn SEO
    • Beginner's Guide to SEO
    • SEO Learning Center
    • Moz Academy
    • SEO Q&A
    • Webinars, Whitepapers, & Guides
  • Blog
  • Why Moz
    • Agency Solutions
    • Enterprise Solutions
    • Small Business Solutions
    • Case Studies
    • The Moz Story
    • New Releases
  • Log in
  • Log out
  • Products
    • Moz Pro

      Your all-in-one suite of SEO essentials.

    • Moz Local

      Raise your local SEO visibility with complete local SEO management.

    • STAT

      SERP tracking and analytics for enterprise SEO experts.

    • Moz API

      Power your SEO with our index of over 44 trillion links.

    • Compare SEO Products

      See which Moz SEO solution best meets your business needs.

    • Moz Data

      Power your SEO strategy & AI models with custom data solutions.

    NEW Keyword Suggestions by Topic
    Moz Pro

    NEW Keyword Suggestions by Topic

    Learn more
  • Free SEO Tools
    • Domain Analysis

      Get top competitive SEO metrics like DA, top pages and more.

    • Keyword Explorer

      Find traffic-driving keywords with our 1.25 billion+ keyword index.

    • Link Explorer

      Explore over 40 trillion links for powerful backlink data.

    • Competitive Research

      Uncover valuable insights on your organic search competitors.

    • MozBar

      See top SEO metrics for free as you browse the web.

    • More Free SEO Tools

      Explore all the free SEO tools Moz has to offer.

    What is your Brand Authority?
    Moz

    What is your Brand Authority?

    Check yours now
  • Learn SEO
    • Beginner's Guide to SEO

      The #1 most popular introduction to SEO, trusted by millions.

    • SEO Learning Center

      Broaden your knowledge with SEO resources for all skill levels.

    • On-Demand Webinars

      Learn modern SEO best practices from industry experts.

    • How-To Guides

      Step-by-step guides to search success from the authority on SEO.

    • Moz Academy

      Upskill and get certified with on-demand courses & certifications.

    • SEO Q&A

      Insights & discussions from an SEO community of 500,000+.

    Unlock flexible pricing & new endpoints
    Moz API

    Unlock flexible pricing & new endpoints

    Find your plan
  • Blog
  • Why Moz
    • Small Business Solutions

      Uncover insights to make smarter marketing decisions in less time.

    • Agency Solutions

      Earn & keep valuable clients with unparalleled data & insights.

    • Enterprise Solutions

      Gain a competitive edge in the ever-changing world of search.

    • The Moz Story

      Moz was the first & remains the most trusted SEO company.

    • Case Studies

      Explore how Moz drives ROI with a proven track record of success.

    • New Releases

      Get the scoop on the latest and greatest from Moz.

    Surface actionable competitive intel
    New Feature

    Surface actionable competitive intel

    Learn More
  • Log in
    • Moz Pro
    • Moz Local
    • Moz Local Dashboard
    • Moz API
    • Moz API Dashboard
    • Moz Academy
  • Avatar
    • Moz Home
    • Notifications
    • Account & Billing
    • Manage Users
    • Community Profile
    • My Q&A
    • My Videos
    • Log Out

The Moz Q&A Forum

  • Forum
  • Questions
  • Users
  • Ask the Community

Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

  1. Home
  2. Research & Trends
  3. White Hat / Black Hat SEO
  4. Screaming Frog tool left me stumped

Moz Q&A is closed.

After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.

Screaming Frog tool left me stumped

White Hat / Black Hat SEO
4
9
1.6k
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as question
Log in to reply
This topic has been deleted. Only users with question management privileges can see it.
  • Ruchy
    Ruchy last edited by Jan 25, 2017, 6:37 PM

    Hi there again,

    I found a major cloaking hack in our client's website that is really well camouflaged and all the seo tools that I tried to help me check for cloaking couldn't find it.

    I know that screaming frog is a great tool and I want to use it to help me, however, I can't seem to get my way around their system that I downloaded.

    Can you help me with the screaming frog program? Do you know where I can make a full site check for cloaking, maybe there are more links that I wasn't notified about?

    I would really appreciate if you could help me with that.

    Thanks so much, Ruchy

    1 Reply Last reply Reply Quote 2
    • BlueprintMarketing
      BlueprintMarketing @Ruchy last edited by Mar 15, 2017, 4:12 AM Mar 15, 2017, 4:12 AM

      Hi Ruchy,

      That's fantastic deep crawl is a great tool.

      I was talking about two different types of cloaking one using an URL with a  C name for hidden cloaking  e.g. https://www.rebrandly.com/

      Thank you for the kind words.  Giving good answers and thumbs up or thumbs down definitely does make a difference I have outlined a lot of it below the official MozPoints URLhttp://moz.com/community/mozpoints

      Yes marking anything as a giving thumbs-up is telling somebody they have helped you and in the event of thumbs up is their response was valuable and maybe contributed to answering your question.

      In the case, of giving good answer person has answered your question giving as well as thumbs-up is the way anything is a way of saying thank you.

      Welcome to the most community, and I look forward to seeing you here.

      All the best,

      Tom

      1 Reply Last reply Reply Quote 0
      • topic:timeago_earlier,about a month
      • Ruchy
        Ruchy last edited by Jan 30, 2017, 1:41 PM Jan 30, 2017, 1:41 PM

        Hi Tom

        we finally solved the issue. Thanks for your clear answers, the info really helped me. We bought DeepCrawl and it really was helpful.

        What was the C name you were talking about?

        Thank you, Ruchy

        P.S. i marked good answer by the wrong one, does it make a difference?

        BlueprintMarketing 1 Reply Last reply Mar 15, 2017, 4:12 AM Reply Quote 1
        • BlueprintMarketing
          BlueprintMarketing last edited by Jan 30, 2017, 1:38 PM Jan 29, 2017, 6:20 PM

          Hey just wanted to check was any answer helpfull to you?

          Is there anything  we might've missed or failed to help you solve the issue?

          All the best,

          Tom

          1 Reply Last reply Reply Quote 1
          • BlueprintMarketing
            BlueprintMarketing @Ruchy last edited by Jan 26, 2017, 8:25 PM Jan 26, 2017, 8:25 PM

            Full site http://www.ghosturl.com/

            or

            https://www.rebrandly.com/

            https://d3e7f5z1blhqw4.cloudfront.net/static/imgs/animation-homepage.gif

            animation-homepage.gif

            1 Reply Last reply Reply Quote 0
            • Ruchy
              Ruchy @Andy.Drinkwater last edited by Jan 26, 2017, 3:39 PM Jan 26, 2017, 3:39 PM

              Thanx Andy

              That was a helpful piece of information.

              I am still a little confused, can you give me some further advice?

              I there any way for me to do a full site cloaking check?

              Ruchy

              BlueprintMarketing 1 Reply Last reply Jan 26, 2017, 8:25 PM Reply Quote 0
              • BlueprintMarketing
                BlueprintMarketing last edited by Jan 26, 2017, 10:17 AM Jan 26, 2017, 10:17 AM

                Hi Ruchy,

                Screaming Frog is an excellent tool. Dan is a great guy to update and the on that.

                In my opinion, I have been able to find formation using https://www.deepcrawl.com/

                The cost is higher deepcrawl is also a hosted platform.

                Still, it can do remarkable things that no other tool that I know of can do.

                https://www.deepcrawl.com/blog/best-practice/12-seo-issues-and-practices-that-google-will-penalize-you-for/

                There are built just for checking cloaking this one is free

                http://www.seotools.com/seo-cloaking-checker/

                If you have access to the DNS will want to examine this thoroughly.

                I have laid out how you would find c name DNS records if you do not have access to the DNS as well.

                One other method you might have to use to determine whether or not domain or subdomain has been cloaked as shown here: http://www.ghosturl.com/  if the cloaking happens in a manner described in the URL for this sentence.

                So the scenario would be the cloaked URL would be something like

                http://www.cloaked.example.com/?=ku2b4B30ijbasT47720sb534Nbq6

                ( please know I used example.com is the domain because I did not want to pointed that any live site not because of anything to do with cloaking)

                Therefore they're using most likely a C name inside the DNS to pointed to  http://www.example.com

                You will need to use a tool that would be able to look at the IP or C name being created by the DNS.

                The free tool I know if that can look for a C name or A record that should not be there is https://www. cloudflare.com the way you would do this is run the site through the DNS wizard that pulls the current DNS including all the records (it can miss some, but it does an excellent job of catching most of all DNS records. (9 times out of 10 will get all of them)

                Because most tools make it impossible to see the C name when looking at DNS, it is important to remember there is a free service that is not designed for this purpose but can be used to discover the rogue cloaked URL.

                https://www.cloudflare.com/a/sign-up (the free-tier offers the same ability to get DNS information)

                http://i.imgur.com/Wm7W0aM.mp4

                https://cl.ly/0A3k3d2j401n/Screen Recording 2017-01-26 at 07.09 AM.mov

                http://i.imgur.com/lIN9Gsm.png

                Hope This helps,

                Tom

                Wm7W0aM.gif lIN9Gsm.png

                1 Reply Last reply Reply Quote 0
                • Andy.Drinkwater
                  Andy.Drinkwater last edited by Jan 26, 2017, 7:05 AM Jan 26, 2017, 7:05 AM

                  Dan from Screaming Frog has just had a look at this, and although he is on the move at the moment, he said this...

                  On the road to a meeting, but they can switch user-agent & change to JavaScript rendering mode to test for cloaking. Depends how it's cloaked, though. Some may do via referrer which you can't test for within the Spider (yet!). Will reply a bit later 🙂

                  -Andy

                  Ruchy 1 Reply Last reply Jan 26, 2017, 3:39 PM Reply Quote 0
                  • Ham1979
                    Ham1979 last edited by Jan 25, 2017, 6:58 PM Jan 25, 2017, 6:58 PM

                    Hi there

                    You can spider the site with screaming frog and then do a bulk export of all the outlinks into a CSV and run through and see if there are links in there that shouldn't be

                    Have you downloaded the latest version?

                    1 Reply Last reply Reply Quote 0
                    • 1 / 1
                    1 out of 9
                    • First post
                      1/9
                      Last post

                    Got a burning SEO question?

                    Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.


                    Start my free trial


                    Browse Questions

                    Explore more categories

                    • Moz Tools

                      Chat with the community about the Moz tools.

                    • SEO Tactics

                      Discuss the SEO process with fellow marketers

                    • Community

                      Discuss industry events, jobs, and news!

                    • Digital Marketing

                      Chat about tactics outside of SEO

                    • Research & Trends

                      Dive into research and trends in the search industry.

                    • Support

                      Connect on product support and feature requests.

                    • See all categories

                    Related Questions

                    • 1kB_man_YWH

                      Does Google only show high rankings to webmasters?

                      google seo

                      Hello! I'm using Google Chrome browser. On the browser logged in with the Google Web Master account, the keyword ranking of my site is ranked on pages 1 to 2. However, in the case of the Google Secret tab and other browsers that have not logged in to the Google Web Master account, the keyword ranking of my site is only 10 pages. Which one is more reliable?
                      Does it only show higher rankings to webmasters?
                      Or will you soon be able to see the high ranking that has only been seen by webmasters in other browsers? I desperately need your help. Thank you. I look forward to your kind cooperation.

                      White Hat / Black Hat SEO | Nov 30, 2024, 4:55 PM | 1kB_man_YWH
                      1
                    • David_Fisher

                      Best practices for retiring 100s of blog posts?

                      redirects blog post backlinks blog post 404s

                      Hi. I wanted to get best practices for retiring an enterprise blog with hundreds of old posts with subject matter that won't be repurposed. What would be the best course of action to retire and maintain the value of any SEO authority from those old blog pages? Is it enough to move those old posts into an archive subdirectory and Google would deprioritize those posts over time? Or would a mass redirect of old blog posts to the new blog's home page be allowed (even though the old blog post content isn't being specifically replaced)? Or would Google basically say that if there aren't 1:1 replacement URLs, that would be seen as soft-404s and treated like a 404?

                      White Hat / Black Hat SEO | Jan 19, 2024, 7:14 AM | David_Fisher
                      0
                    • Arlinaite47

                      Inbound links to internal search with pharma spam anchor text. Negative seo attack

                      Suddenly in October I had a spike on inbound links from forums and spams sites. Each one had setup hundreds of links. The links goes to WordPress internal search. Example: mysite.com/es/?s=⚄

                      White Hat / Black Hat SEO | Apr 6, 2024, 1:26 AM | Arlinaite47
                      0
                    • Ericrodrigo

                      How to fix site breadcrumbs on mobile google search

                      For past one month, I have been doing some research on how to fix this issue on my website but all my efforts didn't work out I really need help on this issue because I'm worried about this I was hoping that Google will cache or understand the structure of my site and correct the error the breadcrumb is working correctly on desktop but not shown on mobile. For Example take a look at : https://www.xclusivepop.com/omah-lay-bad-influence/

                      White Hat / Black Hat SEO | Mar 8, 2024, 5:10 PM | Ericrodrigo
                      0
                    • calvinkj

                      What is the proper URL length? in seo

                      i learned that having 50 to 60 words in a url is ok and having less words is preferable by google. but i would like to know that as i am gonna include keywords in the urls and i am afraid it will increase the length. is it gonna slighlty gonna hurt me? my competitors have 8 characters domain url and keywords length of 13 and my site has 15 character domain url and keywords length of 13 which one will be prefered by google.

                      White Hat / Black Hat SEO | Feb 3, 2024, 8:53 PM | calvinkj
                      0
                    • xaviplabor

                      Malicious backlinks

                      Hello to everyone! We have identified some weird links that are pointing to our site and we are not sure if they are considered malicious backlinks and we should disavow them. Most of them are directories of websites, the most common one is called "Top million domains by alexa" (you can see an example here: www.besafe.in/domain-list-237). Have you ever seen these kind of links before? Are they causing harm to our site? Thank you so much!

                      White Hat / Black Hat SEO | Jun 20, 2024, 9:06 AM | xaviplabor
                      0
                    • ColeLusby

                      Can the disavow tool INCREASE rankings?

                      Hi Mozzers, I have a new client who has some bad links in their profile that are spammy and should be disavowed. They rank on the first page for some longer tail keywords. However, we're aiming at shorter, well-known keywords where they aren't ranking. Will the disavow tool, alone, have the ability to increase rankings (assuming on-site / off-site signals are better than competition)? Thanks, Cole

                      White Hat / Black Hat SEO | Feb 25, 2015, 2:54 PM | ColeLusby
                      0
                    • Heehaw

                      Website not listing in google - screaming frog shows 500 error? What could the issue be?

                      Hey, http://www.interconnect.org.uk/ - the site seems to load fine, but for some reason the site is not getting indexed.   I tried running the site on screaming frog, and it gives a 500 error code, which suggests it can't access the site?  I'm guessing this is the same problem google is having, do you have any ideas as to why this may be and how I can rectify this? Thanks, Andrew

                      White Hat / Black Hat SEO | Nov 7, 2013, 9:47 AM | Heehaw
                      0

                    Get started with Moz Pro!

                    Unlock the power of advanced SEO tools and data-driven insights.

                    Start my free trial
                    Products
                    • Moz Pro
                    • Moz Local
                    • Moz API
                    • Moz Data
                    • STAT
                    • Product Updates
                    Moz Solutions
                    • SMB Solutions
                    • Agency Solutions
                    • Enterprise Solutions
                    Free SEO Tools
                    • Domain Authority Checker
                    • Link Explorer
                    • Keyword Explorer
                    • Competitive Research
                    • Brand Authority Checker
                    • MozBar Extension
                    • MozCast
                    Resources
                    • Blog
                    • SEO Learning Center
                    • Help Hub
                    • Beginner's Guide to SEO
                    • How-to Guides
                    • Moz Academy
                    • API Docs
                    About Moz
                    • About
                    • Team
                    • Careers
                    • Contact
                    Why Moz
                    • Case Studies
                    • Testimonials
                    Get Involved
                    • Become an Affiliate
                    • MozCon
                    • Webinars
                    • Practical Marketer Series
                    • MozPod
                    Connect with us

                    Contact the Help team

                    Join our newsletter
                    Moz logo
                    © 2021 - 2025 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
                    • Accessibility
                    • Terms of Use
                    • Privacy

                    Looks like your connection to Moz was lost, please wait while we try to reconnect.