Is real google bot like "fetch"or more like "fetch & render"?
-
In GWT we have two options to mimic googlebot visits, "fetch" and "fetch and render", but when the real googlebot visit a page, is he behaving like the former or the latter?
I can see fetch does fetch only the html, while fetch and render does fetch .js and .css as well.
But what does the real googlebot does?
I have checked the web server logs, and I can see the real googlebot sometimes request the .js files too, but not every time it visit a page, sometimes it does, sometimes it does not.
Has anyone figured out when googlebot actually request javascript files?
-
That was my guess, that google somehow cache the .js files, so I changed one javascript file name to a random generated name. Did it last night, I will recheck logs in a couple of days to see what happen.
-
Googlebot is behaving like the latter, "Googlebot constantly crawls and renders URLs across the entire web..." from https://support.google.com/webmasters/answer/6066467, but has the option of streamlining the process by not rendering every time. For example, if many sites are referencing the same centrally hosted code base of js, googlebot doesn't need to constantly recheck that js every time it comes across the link.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Increase in Direct traffic, drop in Google traffic, after updating Google Analytics settings.
I made some changes to our Google Analytics property settings (see notes in screenshot). As a result, there was an equal drop in Google traffic and increase in Direct traffic. Has anyone else seen something like this before? I'm wondering if I should revert. jE7buO6
Reporting & Analytics | | vcj0 -
Google Analytics Goal/Event/SOMETHING to show only Wordpress "Posts", not pages, etc
Hi all, Our site is build on Wordpress and formerly the post URL's had the typical date format at the beginning. This made it easy for me to look at, for example, all search traffic to the blog. I would just view URL's containing /2014/ and /2015/ and boom. We have since removed the dates from the URL's with proper redirects etc, which is great, but now I can't figure out a way to look at ONLY the blog in GA. I like to track a KPI of 'search visits to blog posts' and I can't figure out how to now. Can I set up a GA event that only fires when the post type template for blog posts loads? Some other solution? I'm lost here, and there's gotta be a good way to do it...
Reporting & Analytics | | 3DR0 -
Google penalty
Hi, I've seen a steady improvement in my google search referrals since optimising my site and keeping an eye on SEO over the last few months. I'm only getting relatively small numbers of referrals, around 120-130 a day, but this has steadily increased from about 40 before Christmas. It's been a fairly consistent increase until 4 days ago when suddenly my referral numbers were cut in half. I'd be surprised if google was penalising me since I'm such a small site, but are there any obvious signs I should look out for? Oh, and my site is www.madegood.org should that be of help. Thanks! Will
Reporting & Analytics | | madegood0 -
Mobile Site on Google Analytics
Hi mozzers, We just launched a mobile site and I was wondering what are the main steps to follow for gettting your mobile site tracked via GA (m.example.com)? We have a profile for www.example.com GATC: javascript or PHP to install? Should the profile be on a subdomain? What else to consider when implementing a mobile site on GA? Thanks
Reporting & Analytics | | Ideas-Money-Art0 -
Traffic Drop with no real explaination?
Hello Everyone, On one of my sites PokerOnaMac we experienced a 75% drop in traffic in about a week (since September 3rd). I took a look at GA and the penalties have been applied sitewide. I also checked my Google Webmaster Tools, and there are no messages regarding "unhealthy link structures." We have had other sites penalized by Penguin & Manual Removals, but they are recovering now. What is my best bet to find what is wrong? Thanks
Reporting & Analytics | | BestOdds0 -
Using Regex for Goals in Google Analytics
I’ve got a website with 20 different forms; and I’d like to track all 20 form completions with one Goal in Google Analytics. The completion-page URLs follow the pattern below (the website uses Drupal CMS) Form One: www.acme.com/node/2990/done?sid=651 Form Two: www.acme.com /node/2991/done?sid=785 Form Three: www.acme.com /node/2992/done?sid=1021 Form Four: www.acme.com /node/2993/done?sid=459 I believe that there is a way I can use regex so Google Analytics will track if any of these completion URLs appears. Looking for guidance how to format the regex statement.
Reporting & Analytics | | TopFloor0 -
Where are google analytics stats for iphone4
hi We were looking at the Google Analytics for one of our sites and noticed that there were NO pageviews from device=iphone and resolution=640x960 in the report. Given that iphone4 is supposed to be 640x960, and would be the most popular device (at least in our offices and everyone I know), it seems wierd. I sorted the Mobile Devices report by device and resolution to see what was available. The first 160 results were all device=not set. Finally got to device=iPhone and there were three entries: resolution 0x0 had 11 views resolution 320x396 had 45 views resolution 320x480 had 3,944 views. Hopefully all iphone4 users havent been classified as not set. Or is it possible that iphone4s claim to be 320x480 in browsers, as per http://www.alistapart.com/articles/a-pixel-identity-crisis/ Even worse, if I look at the Samsung Galaxy S II (myown phone), there are over 30 screen resolution combinations. Does anyone have anything to shed on this? I asked about it on the google analytics twitter account last week but havent had a response. Are there other analytics solutions that would distinguish between the iphones? Warning - this is a link to a large image, with the not set stats at the top. 6Sjji
Reporting & Analytics | | ozgeekmum0