What referrer is shown in http request when google crawler visit a page?
-
Is it legit to show different content to http request having different referrer?
case a: user view one page of the site with plenty of information about one brand, and click on a link on that page to see a product detail page of that brand, here I don't want to repeat information about the brand itself
case b: a user view directly the product detail page clicking on a SERP result, in this case I would like to show him few paragraph about the brand
Is it bad? Anyone have experience in doing it?
My main concern is google crawler. Should not be considered cloaking because I am not differentiating on user-agent bot-no-bot.
But when google is crawling the site which referrer will use? I have no idea, does anyone know?
When going from one link to another on the website, is google crawler leaving the referrer empty?
-
After googling about it and thinking better I choose not too. I think you are absolutely right, too inconsistent.
-
You won't get a referrer. Googlebot is not like a real user, surfing from site to site. What Googlebot does is this (more or less)
- Googlebot requests a standalone page
- Googlebot parses the page out. During this process it notes the links on that page and, depending on various mechanisms (nofollow, internal page rank, the mood of Matt Cutts, etc) it will note those links for the system to parse later
- Googlebot is done so it grabs another page off the page list (likely without know how it got on said list) and goes back to #1
Now, to your question. Since Googlebot has no referrer it won't get your alternate content. This means that your alternate content page won't get indexed.
I would suggest here that a best practice is NOT to filter on referrer data, which can be inconsistent and potentially fake. Instead, I would make a separate page that contains your extra data and allow users to decide if they want more information Thus Googlebot finds all your content and your users get a better experience.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Stuck on the 2nd page of google! Help
I run a McAfee Technical Support website. I has been 2.3 months since I have been practicing seo on it. It was slick until it appeared on the second page of google. But now it doesnt rank up as it's frozen. Can i get any advices and suggestions for my website to break the 2nd page cage. My website:-** mcafee.com/activate**
Intermediate & Advanced SEO | | six_figures0 -
Different snippet in Google for same page
Hello, I have a question regarding the snippet of a specific case: When i search the homepage by searching the business name, i find the correct snippet of the homepage (with the meta description that was entered). If i search it via site:www. it still show the default meta description. Has anybody had experience with this? Is there a way to change the snippet of site:www.? Does it influence SEO? Thank you!
Intermediate & Advanced SEO | | conversal0 -
SEO Impact & Google Impact On Removing Product From Category Page for Ecommerce Site
Hello Experts, For my Ecommerce site previously I was showing products at category pages i.e. first all subcategories name after that list all products of all subcateogries. That also approx per category 500 products via load more feature. My query is now I am planning to show products only at Product Listing Page and not on Category pages so what will be SEO impact and how google will treat this? Thanks!
Intermediate & Advanced SEO | | Johny123450 -
May integrating my main category page in the index page improve my ranking of main category keyword?
90% of our sales are made with products in one of our product categories.
Intermediate & Advanced SEO | | lcourse
A search for main category keyword returns our root domain index page in google, not the category page.
I was wondering whether integrating the complete main category directly in the index page of the root domain and this way including much more relevant content for this main category keyword may have a positive impact on our google ranking for the main category keyword. Any thoughts?1 -
Indexed Pages Different when I perform a "site:Google.com" site search - why?
My client has an ecommerce website with approx. 300,000 URLs (a lot of these are parameters blocked by the spiders thru meta robots tag). There are 9,000 "true" URLs being submitted to Google Search Console, Google says they are indexing 8,000 of them. Here's the weird part - When I do a "site:website" function search in Google, it says Google is indexing 2.2 million pages on the URL, but I am unable to view past page 14 of the SERPs. It just stops showing results and I don't even get a "the next results are duplicate results" message." What is happening? Why does Google say they are indexing 2.2 million URLs, but then won't show me more than 140 pages they are indexing? Thank you so much for your help, I tried looking for the answer and I know this is the best place to ask!
Intermediate & Advanced SEO | | accpar0 -
Why is Google Ranking the Umbrella Category Page when Searching for Sub-Categories Within that Umbrella Category?
I have an e-commerce client who sells shoes. There is a main page for "Kids" shoes, and then right under it on the top-navigation bar there is a link to "Boys Shoes" and "Girls Shoes." All 3 of these links are on the same level - 1 click off the home page. (And linked to from every page on the website via the top nav bar). All 3 are perfectly optimized for their targeted term. However, when you search for "boys shoes" or "girls shoes" + the brand, the "Kids" page is the one that shows up in the #1 position. There are sitelinks beneath the listing pointing to "Girls" and "Boys." All the other results in Google are resellers of the "brand + girls" or "brand + boys" shoes. So our listing is the only one that's "brand + kids shoes." Our "boys" shoes page and "girls" shoes page don't even rank on the 1st page for "brand + boys shoes" or "brand + girls shoes." The only real difference is that "kids shoes" contains both girls and boys shoes on the page, and then "boys" obviously contains boys' shoes only, "girls" contains girls' shoes only. So in that sense there is more content on the "kids" page. So my question is - WHY is the kids page outranking the boys/girls page? How can we make the boys/girls pages be the ones that show up when people specifically search for boys/girls shoes?
Intermediate & Advanced SEO | | FPD_NYC0 -
To land page or not to land page
Hey all, I wish to increase my sites rankings on a variety of keywords within sub categories but I'm unsure where to be spending the time in SEO. Here's an example of the website page structure: General Home Page > Sub Category 1 Home Page
Intermediate & Advanced SEO | | DPSSeomonkey
> Searching / Results pages
- Sub Category 1
- Sub Category 2
- Sub Category 3
- Sub Category 4 > Sub Category 2 Home Page
> Searching / Results pages
- Sub Category 1
- Sub Category 2
- Sub Category 3
- Sub Category 4 We've newly introduced the Sub Category Home Pages and I was wondering if SEO is best performed on these pages or should landing pages be built, one for each of the 4 sub categories in each section. Those landing pages would have links to the "Searching / Results pages" for that sub category. Thanks!0 -
Google consolidating link juice on duplicate content pages
I've observed some strange findings on a website I am diagnosing and it has led me to a possible theory that seems to fly in the face of a lot of thinking: My theory is:
Intermediate & Advanced SEO | | James77
When google see's several duplicate content pages on a website, and decides to just show one version of the page, it at the same time agrigates the link juice pointing to all the duplicate pages, and ranks the 1 duplicate content page it decides to show as if all the link juice pointing to the duplicate versions were pointing to the 1 version. EG
Link X -> Duplicate Page A
Link Y -> Duplicate Page B Google decides Duplicate Page A is the one that is most important and applies the following formula to decide its rank. Link X + Link Y (Minus some dampening factor) -> Page A I came up with the idea after I seem to have reverse engineered this - IE the website I was trying to sort out for a client had this duplicate content, issue, so we decided to put unique content on Page A and Page B (not just one page like this but many). Bizarrely after about a week, all the Page A's dropped in rankings - indicating a possibility that the old link consolidation, may have been re-correctly associated with the two pages, so now Page A would only be getting Link Value X. Has anyone got any test/analysis to support or refute this??0