How much content does Google Crawl on your site?
-
Hi,
We've had a debate around the office where some people believe that Google only crawls the first 150-200 words on a page and some people believe that they priority content that is above the fold and other people believe that all content has the same priority. Can you help us?
Thanks,
Matt -
Google actually crawls 150kb, excluding css files, images, etc.
150kb is much more than 200 words, and the experiment suggested by Mr Bennett proves it.
-
They definitely crawl more than that, and it's easy to prove as well.
Pick a long page, such as the Wikipedia page about London. Choose a block of text from near the bottom of that page, I've selected this:
in the south-western suburb of Wimbledon.[252] Other key events are the annual mass-participation London Marathon which sees some 35,000 runners
If you search for that text you will see the Wikipedia page in the results. If they only crawled the first 200 words they wouldn't have been able to find that result.
Prioritising is harder to demonstrate (and probably also to define!). However it is generally believe that greater importance is given to text towards the top of the page. That is logical if you consider how the majority of documents are structured.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google showing my content on the serps in a different domain
Hi all, Recently a partner of ours discovered that Google is showing a meta description on the serps for his homepage that is not his but ours. On his site, he sells add-ons for our software, so the name of our software appears many times and as well there are many links pointing to our site. He claims he hasn´t copied this text from us, and I have used some tools to verify this. I don´t understand how Google can get confused and show our text as the meta desctiption on the serps for his homepage. Any idea on why this happened?
On-Page Optimization | | Paessler0 -
Duplicate content on domains we own
Hello! We are new to SEO and have a problem we have caused ourselves. We own two domains GoCentrix.com (old domain) and CallRingTalk.com (new domain that we want to SEO). The content was updated on both domains at about the same time. Both are identical with a few exceptions. Now that we are getting into SEO we now understand this to be a big issue. Is this a resolvable matter? At this point what is the best approach to handle this? So far we have considered a couple of options. 1. Change the copy, but on which site? Is one flagged as the original and the other duplicate? 2. Robots.txt noindex, nofollow on the old one. Any help is appreciated, thanks in advance!
On-Page Optimization | | CallRingTalk0 -
How to use canonical with mobile site to main site
I am pretty sure that the mobile version of the main site needs to be the same canonical link from what I understand. I am trying to find good docuementation that supports this. Even better if its from Google or Matt Cutts. I have a main domain like http://www.mydomain.com the mobile version of this is http://www.mydomain.com/m/ Should my canonical be rel="canonical" href="http://www.mydomain.com"/> for both these pages?
On-Page Optimization | | cbielich0 -
Can you have more than 1 site on the first page if site look and content is completely different but keywords are the sam.
I have a client that wants to build another completely different site than his main site and optimize it to have 2 websites on the first page for his keywords. The content and look and feel of the website would be completely different. One of his competitors is doing it and getting away with it. What is your advice.
On-Page Optimization | | Roots70 -
Site: command and intitle: command in Google changed?
Hi Mozzers, I'm seeing some changes in Google when using certain commands I've used for ages. I'm trying to spot cananical issues by using this search site:www.mysite.com intitle:"keyword" This used to list all pages in the index on a certain site with the keyword in the title. Now I'm getting weird results and sometimes results from other sites - not the one specified in the site: command. Anyone else seeing this? Thanks B
On-Page Optimization | | Bush_JSM0 -
Google Instant Preview
Is there a way of having videos show up in google instant preview? Right now all I am getting is a blank space
On-Page Optimization | | casper4340 -
Duplicate Content
Hi I have Duplicate content that i do sent understand 1 - www.example.dk 2- www.example.dk/ I thought i was the same page, whit and without the / Hope someone can help 🙂
On-Page Optimization | | seopeter290 -
Crawling - Blue Notice - Canonical
Hi, I have 270x blue notices within crawl diagnostics in SEOMoz Pro labelled rel=canonical. My site has the rel=canonical tag set-up as I was advised to do so. See www.comparecurrency.co.uk Are these notices suggesting I have to remove the tag? Can somebody please explain this notice to me .. Thanks Olly
On-Page Optimization | | ojkingston0