Is real google bot like "fetch"or more like "fetch & render"?
-
In GWT we have two options to mimic googlebot visits, "fetch" and "fetch and render", but when the real googlebot visit a page, is he behaving like the former or the latter?
I can see fetch does fetch only the html, while fetch and render does fetch .js and .css as well.
But what does the real googlebot does?
I have checked the web server logs, and I can see the real googlebot sometimes request the .js files too, but not every time it visit a page, sometimes it does, sometimes it does not.
Has anyone figured out when googlebot actually request javascript files?
-
That was my guess, that google somehow cache the .js files, so I changed one javascript file name to a random generated name. Did it last night, I will recheck logs in a couple of days to see what happen.
-
Googlebot is behaving like the latter, "Googlebot constantly crawls and renders URLs across the entire web..." from https://support.google.com/webmasters/answer/6066467, but has the option of streamlining the process by not rendering every time. For example, if many sites are referencing the same centrally hosted code base of js, googlebot doesn't need to constantly recheck that js every time it comes across the link.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to activate google optimize free version?
HI, How to activate https://www.google.com/analytics/optimize/ free for my ecommerce site? I am already using google analytic and google tag manager? So this is best tool for doing A/B testing? It is free for 30 days only? Thanks
Reporting & Analytics | | dsouzac0 -
Have we been penalised by Google?
Hello guys, Looking for some help here hope someone will be able to point us to the right direction, we have been scratching our heads for months! Basically our Google rankings and organic traffic from Google have dropped signficantly in the last 12 months. At one point we ranked as high as page 3 with the term "holiday lettings". Now we are on page 16, and sometimes not found at all... This has been the case for months now. Curiously, Bing and Yahoo have been much more generous to us, we have been and still consistently rank at No. 2 with the same term. We are in the vacation rental industry - our website is www.alphaholidaylettings.com and most of the sites that rank above us are of much less relevancy, quality and authority. We just could not understand why we would rank so poorly on Google (both US and UK). Organic traffic has also dropped by more than 50% steadily in the last 12 months. One thing we thought might have affected our site is the quality of content. Most content/listings on our site are user generated, and it is not uncommon for users to copy and paste details from an existing listing that they have on another site. Having said that, the same would apply to our competitors too. In this case, might Panda or Penguin have had any negative effects on our seo? We have been adding unique content to the site in the form of a blog and "Holiday Feature" articles. Could the actual design/structure/coding of our pages contribute to our poor rankings too? Any opinions or feedback is much appreciated! Thanks guys, Tom
Reporting & Analytics | | forgottenlife0 -
Google is just plain confusing now
I know, many people are up in arms with Google with their very frequent recent changes. I guess some of this is good - but at times I am also warming to the opinion that they are just losing the plot. To illustrate my point - check this ranking history for a keyword: Toyota South Africa I'm not sure how this image will display - but for no obvious apparent reason, from 02/10 - we were ranked 5, and now on 9/10 dropped right down to 44. I mean how is on supposed to explain, and rectify this when Google just keeps on changing the playing fields? shrug Ranking.png
Reporting & Analytics | | ZakD0 -
Cross domain tracking setup in Google Analytics
I really need help on this, I can't figure it out. I need to track a few subdomains and and couple of seperate domains in one profile in Google Analytics. So I have: www.maindomain.co.uk sub1.maindomain.co.uk sub2.maindomain.co.uk www.anotherdomain.co.uk So the code I have placed on all of these domains is as following: I have then set up seperate profiles which filter out the data by domain. E.g. for sub1.maindomain.co.uk, I have user the fitler: Custom filter > Include > Filter Field = Hostname > Filter Pattern = sub1.maindomain.co.uk For the www.anotherdomain.co.uk, I have used the same filter. Now this all seems to be tracking visits and unique visitors okay, but I always have a very low pageview count, less than visits and sometimes zero. Could anyone shed any light on what I am doing wrong? Thanks in advance mozzers.
Reporting & Analytics | | MirandaP0 -
What to get from google Webmaster tools?
Hi everyone, I've been doing optimization for our websites and tracking the results regularly but don't really know what the results actually mean. I heard that I need to check the traffic and organic results from Google analytics and Google webmaster tools. Everybody says something and not sure what to do? Is there anyone clarify the SEO process for me - when to do what and why in simplest way? Thanks in advance,
Reporting & Analytics | | WTGEvents0 -
Google Analytics session update question
Hello, With reference to seomoz blog post - http://www.seomoz.org/blog/panda-24-and-analytics-session-update-rolled-out-simultaneously#jtc151292 , i would like clarification about the following - User searches Google for "Product Name" and clicks on your AdWords advertisement. User leaves site and searches a few more times, click on competition and comparing prices and features. User ultimately decides to with your product, Googles "Your Brand + Product Name", clicks your organic listing, and buys the product. This whole process takes less than 30 minutes. "Your Brand + Product Name" will appear in your organic keyword report with 1 visit. My question is whether "Product Name" will also appear in organic keyword report with 1 visit if the visitor is not signed in. ( as search won't be encrypted ) Thanks
Reporting & Analytics | | seoug_20100 -
Google Analytics Best Practice Set up for Clients
Hi When setting up new Google Analytics accounts for clinets what is the preferred/best practice. At present we have our own company google account and add new clinets this way (to our account) - the disadvantage with this, we can only grant them limited account access otherwise they would be able to view all the accounts we cretaed. Plus we can't link their adwords to the GA account we cretaed them. Is it best practice to set the client up with their own Google Account and then we just link to their account. Advise would be appreciated, thank you.
Reporting & Analytics | | daracreative0 -
Do Google Analytics filters affect link building?
We recently made a few link wheels for specific product pages. We've been having great results with all of the wheels except for one. The one we are having issues with is the only link that we were using a Google Analytics filter on; it looks like this http://domain.com/page.htm?zSource=Specific Keyword%tracking My question is does Google ignore links that are obviously utilizing their analyitcs custom filters? We're doing some more testing to try to find out if it truly is the link or if the wheel is a bad one. There are so many things that could go wrong but with so many of our link wheels working well, I wonder if the filters are what is causing such results.
Reporting & Analytics | | MichealGooden0