The Moz Q&A Forum

    • Forum
    • Questions
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Forum
    2. Categories
    3. SEO Tactics
    4. Intermediate & Advanced SEO
    5. What referrer is shown in http request when google crawler visit a page?

    Moz Q&A is closed.

    After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.

    What referrer is shown in http request when google crawler visit a page?

    Intermediate & Advanced SEO
    3 2 4.1k
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • max.favilli
      max.favilli last edited by

      Is it legit to show different content to http request having different referrer?

      case a: user view one page of the site with plenty of information about one brand, and click on a link on that page to see a product detail page of that brand, here I don't want to repeat information about the brand itself

      case b: a user view directly the product detail page clicking on a SERP result, in this case I would like to show him few paragraph about the brand

      Is it bad? Anyone have experience in doing it?

      My main concern is google crawler. Should not be considered cloaking because I am not differentiating on user-agent bot-no-bot.

      But when google is crawling the site which referrer will use? I have no idea, does anyone know?

      When going from one link to another on the website, is google crawler leaving the referrer empty?

      1 Reply Last reply Reply Quote 0
      • max.favilli
        max.favilli @Highland last edited by

        After googling about it and thinking better I choose not too. I think you are absolutely right, too inconsistent.

        1 Reply Last reply Reply Quote 0
        • Highland
          Highland last edited by

          You won't get a referrer. Googlebot is not like a real user, surfing from site to site. What Googlebot does is this (more or less)

          1. Googlebot requests a standalone page
          2. Googlebot parses the page out. During this process it notes the links on that page and, depending on various mechanisms (nofollow, internal page rank, the mood of Matt Cutts, etc) it will note those links for the system to parse later
          3. Googlebot is done so it grabs another page off the page list (likely without know how it got on said list) and goes back to #1

          Now, to your question. Since Googlebot has no referrer it won't get your alternate content. This means that your alternate content page won't get indexed.

          I would suggest here that a best practice is NOT to filter on referrer data, which can be inconsistent and potentially fake. Instead, I would make a separate page that contains your extra data and allow users to decide if they want more information Thus Googlebot finds all your content and your users get a better experience.

          max.favilli 1 Reply Last reply Reply Quote 1
          • 1 / 1
          • First post
            Last post

          Got a burning SEO question?

          Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.


          Start my free trial


          Explore more categories

          • Moz Tools

            Chat with the community about the Moz tools.

            Getting Started
            Moz Pro
            Moz Local
            Moz Bar
            API
            What's New

          • SEO Tactics

            Discuss the SEO process with fellow marketers

            Content Development
            Competitive Research
            Keyword Research
            Link Building
            On-Page Optimization
            Technical SEO
            Reporting & Analytics
            Intermediate & Advanced SEO
            Image & Video Optimization
            International SEO
            Local SEO

          • Community

            Discuss industry events, jobs, and news!

            Moz Blog
            Moz News
            Industry News
            Jobs and Opportunities
            SEO Learn Center
            Whiteboard Friday

          • Digital Marketing

            Chat about tactics outside of SEO

            Affiliate Marketing
            Branding
            Conversion Rate Optimization
            Web Design
            Paid Search Marketing
            Social Media

          • Research & Trends

            Dive into research and trends in the search industry.

            SERP Trends
            Search Behavior
            Algorithm Updates
            White Hat / Black Hat SEO
            Other SEO Tools

          • Support

            Connect on product support and feature requests.

            Product Support
            Feature Requests
            Participate in User Research

          • See all categories

          • Google Indexing Of Pages As HTTPS vs HTTP
            vikasnwu
            vikasnwu
            1
            13
            4.3k

          Get started with Moz Pro!

          Unlock the power of advanced SEO tools and data-driven insights.

          Start my free trial
          Products
          • Moz Pro
          • Moz Local
          • Moz API
          • Moz Data
          • STAT
          • Product Updates
          Moz Solutions
          • SMB Solutions
          • Agency Solutions
          • Enterprise Solutions
          • Digital Marketers
          Free SEO Tools
          • Domain Authority Checker
          • Link Explorer
          • Keyword Explorer
          • Competitive Research
          • Brand Authority Checker
          • Local Citation Checker
          • MozBar Extension
          • MozCast
          Resources
          • Blog
          • SEO Learning Center
          • Help Hub
          • Beginner's Guide to SEO
          • How-to Guides
          • Moz Academy
          • API Docs
          About Moz
          • About
          • Team
          • Careers
          • Contact
          Why Moz
          • Case Studies
          • Testimonials
          Get Involved
          • Become an Affiliate
          • MozCon
          • Webinars
          • Practical Marketer Series
          • MozPod
          Connect with us

          Contact the Help team

          Join our newsletter
          Moz logo
          © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
          • Accessibility
          • Terms of Use
          • Privacy

          Looks like your connection to Moz was lost, please wait while we try to reconnect.