Is real google bot like "fetch"or more like "fetch & render"?
-
In GWT we have two options to mimic googlebot visits, "fetch" and "fetch and render", but when the real googlebot visit a page, is he behaving like the former or the latter?
I can see fetch does fetch only the html, while fetch and render does fetch .js and .css as well.
But what does the real googlebot does?
I have checked the web server logs, and I can see the real googlebot sometimes request the .js files too, but not every time it visit a page, sometimes it does, sometimes it does not.
Has anyone figured out when googlebot actually request javascript files?
-
That was my guess, that google somehow cache the .js files, so I changed one javascript file name to a random generated name. Did it last night, I will recheck logs in a couple of days to see what happen.
-
Googlebot is behaving like the latter, "Googlebot constantly crawls and renders URLs across the entire web..." from https://support.google.com/webmasters/answer/6066467, but has the option of streamlining the process by not rendering every time. For example, if many sites are referencing the same centrally hosted code base of js, googlebot doesn't need to constantly recheck that js every time it comes across the link.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Conflicting data in Google Analytics
Hi Guys I've been looking at the data for a client in Google analytics and I was wondering if anyone knows why some of the data doesn't tally up. In my case its the following: Under Aquisition
Reporting & Analytics | | Relative
No of Sessions for a Keyword shown in Organic Search tab (compared to)
No of Clicks for a Query in the Search Engine Optimisation tab For example, for a brand term, Google are showing 17 Sessions in Organic Search.
For the same term Google are showing 90 Clicks in the Queries section of Search Engine Optimisation OK, we know that Google are a little cloak and dagger regarding keyword data but surely Sessions and Clicks for the same keyword should be identical unless I'm missing something.0 -
Google News Bug
During today and yesterday we are getting a lot of visits (Over 400) from http://news.google.com the visits are all at the same time and to different pages, the visits are very brief so much so that it pushes our bounce rate up to 69% in GA. Is there anything we can do to stop this? Thank you in advance?
Reporting & Analytics | | Palmbourne0 -
Google Analytics Real Time Not Working! :(
Hello Everyone, Today, the real time feature in my google analytics stopped working. I am able to see that traffic is visiting my site, but not in real time. The real time count is usually at 0. But, there are some instances where real time will come back online, but there will be only 1 unique visitor. At any given time, our website usually has 20 visitors or so..Has anyone encountered this issue? Where should i start looking for fixes? What are the possible problems? Thanks!
Reporting & Analytics | | KarlMarxTheBear0 -
Has anyone experienced Google Analytics track the page visit to a "thank you" page, but not the goal conversion?
Has anyone experienced where Google Analytics would track the page visit to a "thank you" page, but not the goal conversion that should result? The goal had worked for a long time as it is as just a goal url with head match. No funnel. Not case sensitive. For about four days now, no conversions have been recorded, but Google Analytics shows hundreds of people visited the page that should trigger the goal. Additionally, we have received the hundreds of leads. A Screaming Frog search shows the code is embedded throughout the site. For the interested, the GA code looks like (and the 8 Xs are the correct number on the site): Am I missing something?
Reporting & Analytics | | 352inc0 -
Google Webmaster Tools not displaying all backlinks
I was asked to take a look at a site that a friend redesigned and immediately plummeted in rankings. It was a flash site where the domain redirected to a flash landing page. When the site was redesigned, they removed the flash page and put in the new site on the domain URL without changing the redirect or putting a 301 in place on the old flash landing page URL. This site didn't have many backlinks to begin with, but it did ok for some important search phrases. Now when I look in Webmaster Tools, there are only 2 backlinks showing. Meanwhile I know there are more backlinks in existence, primarily from yellowpages and citysearch. I've had them put a 301 on the old flash landing page pointing at the domain URL and they've added a canonical URL. What have I missed here?
Reporting & Analytics | | BostonWright0 -
What to get from google Webmaster tools?
Hi everyone, I've been doing optimization for our websites and tracking the results regularly but don't really know what the results actually mean. I heard that I need to check the traffic and organic results from Google analytics and Google webmaster tools. Everybody says something and not sure what to do? Is there anyone clarify the SEO process for me - when to do what and why in simplest way? Thanks in advance,
Reporting & Analytics | | WTGEvents0 -
How Google handles keywords and punctuation?
How does Google handle punctuation of keywords within its search? If I have three keywords together, but on the site there is a question mark between them, does Google ignore the question mark? Or should I think of a new way of writing the content so Google likes it more? And would this apply with articles (a, the, etc) and prepositions such as (in, of, etc.)? Thanks a lot.
Reporting & Analytics | | Rocket.Fuel1 -
Google Analytics: Do url redirects show up
Good Morning from 6 degrees c mostly cloudy wetherby UK 🙂 A redirect has been set up with the effect that when you key in www.hyload.co.uk it forwards to www.ikogroup.co.uk My question is please...within Google analytics will referall traffic from www.hyload.co.uk show up in referral traffic or because its a redirect will it be counted as direct traffic. Thanks in advance, David
Reporting & Analytics | | Nightwing0