How much content does Google Crawl on your site?
-
Hi,
We've had a debate around the office where some people believe that Google only crawls the first 150-200 words on a page and some people believe that they priority content that is above the fold and other people believe that all content has the same priority. Can you help us?
Thanks,
Matt -
Google actually crawls 150kb, excluding css files, images, etc.
150kb is much more than 200 words, and the experiment suggested by Mr Bennett proves it.
-
They definitely crawl more than that, and it's easy to prove as well.
Pick a long page, such as the Wikipedia page about London. Choose a block of text from near the bottom of that page, I've selected this:
in the south-western suburb of Wimbledon.[252] Other key events are the annual mass-participation London Marathon which sees some 35,000 runners
If you search for that text you will see the Wikipedia page in the results. If they only crawled the first 200 words they wouldn't have been able to find that result.
Prioritising is harder to demonstrate (and probably also to define!). However it is generally believe that greater importance is given to text towards the top of the page. That is logical if you consider how the majority of documents are structured.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Duplicate Content
When I say duplicate content, I don't mean that the content on a clients site is displaying on another site on the web or taken from a site on the web. A client has a few product pages and each product page has content on the bottom of the page (4-5 paragraphs) describing the product. Now, this content is also displaying on other pages, but re-worded so it's not 100% duplicate. Some pages show a duplicate content % ranging from 12% to 35% and maybe 40%. Just curious if I should suggest having each product page less than 10% duplicated. Thanks for your help.
On-Page Optimization | | Kdruckenbrod0 -
Responsive site.com vs m.site.com
Hi All, My client's website have two urls like: site.com/a.html and **m.site.com/a.html. ** Will it hurt google rankings for this website because there are version of a website? Please help!
On-Page Optimization | | binhlai1 -
Google Rich Snippet
So i have been implementing rich snippets for work and all has been good until now, As you can see below the meta description has all of a sudden included the review date. The review date is the only date on the page. Any ideas what could be causing this?
On-Page Optimization | | David-McGawnThanks wqLKKl9
0 -
Inbound Linking from your own sites
Good evening, On each of the sites I have made, I have a link with the anchor text 'Build and Design by Christoper Davies' to my own website. This link is in the footer of every page each of all the sites. Should I have a 'no follow' rel added to these links, or does linking from all the sites (on all pages) help my ranking? I am concerned that having so many inbound links from the same sites, with the same anchor text may be doing me more damage than good.
On-Page Optimization | | chrisdavieswebdesign0 -
Should I change PDF content?
Hi everybody, My Website is ranking well for several keywords and long-tail keywords. However, all these visits are going directly to some .PDF guides that exist on our products and information on industry sectors the company is based around. I feel the PDF's are bad simply because they dont offer easy interaction with the rest of the website. I am considering making each PDF into a webpage but am not 100% sure of the pro's and cons of doing so. I will still need to the PDF's accessible for user to download but don't want my new webpages to get tagged as duplicate content. Is it possible to,
On-Page Optimization | | ATP
1 - change the PDF's so they send any link authority to the new webpage
2 - make google aware that I want the webpage not the PDF to be the "ranking" page What is the likely hood of destroying my rank for these keywords on the PDF by making these changes and then not being able to rank the webpage for the same keywords? It would be pointless if I just lost all the traffic lol.0 -
Tool To Search For Duplicate Content
Hi Is there such a tool that can be use to search a website for duplicate content? Thanks
On-Page Optimization | | Bossandy0 -
I have more pages in my site map being blocked by the robot file than I have being allowed to be crawled. Is Google going to hate me for this?
Using some rules to block all pages which start with "copy-of" on my website because people have a bad habit of duplicating new product listings to create our refurbished, surplus etc. listings for those products. To avoid Google seeing these as duplicate pages I've blocked them in the robot file, but of course they are still automatically generated in our sitemap. How bad is this?
On-Page Optimization | | absoauto0 -
Google Indexing
Hi, We recently launched a new version of our site on the Magento platform. I submitted a new sitemap and on the first crawl only 7 pages out of 132 were indexed...a few days later and we now have 107 indexed (phew). My question is this....how on earth do i find out which pages are indexed and more importantly not indexed? For all i know they might be really important ones so I need to be able to identify the missing pages so i can work on getting them indexed. Nic
On-Page Optimization | | nicc19760