Get Google To Crawl More Pages Faster on my Site
-
We opened our database of about 10 million businesses to be crawled by Google.
Since Wednesday, Google has crawled and indexed about 2,000 pages. Google is crawling us at about 1,000 pages a day now. We need to substantially increase this amount.
Is it possible to get Google to crawl our sites at a quicker rate?
-
Mark If you have had your category level or city/state level pages, whatever your architecture is, you'd see your detail pages getting crawled pretty quickly. Give it a week. I would also work on getting some deep links to your website. Google has mentioned several times, PR is a direct measure of how deep and how often Google crawls your website. So I would suggest working on building the domain authority and brand in terms of backlinks to the website. Other then that, as bstone81 said, generate sitemaps and sitemap of sitemaps and submit to Google.
-
Two things you can do:
1. Submit a site map with all those pages. They suggest breaking the sitemaps into 50,000 page groups.
2. Use the "Fetch as Google" feature in GWT and fetch your root domain then click the box to submit the root domain along with all the linked pages. This will submit the request to Google to be indexed.
-
Hi Mark,
Google only spends a certain amount of time to crawl and there is no way to tell them to crawl more or faster. If you have a lot of pages, it might take longer.
One way to make them crawl faster may be optimize your internal linking. If you have links pointing to other pages in your website, it makes it easier to crawl.
Other than work arounds, there are no ways to tell Google to crawl faster or more pages. They have their own speed.
http://support.google.com/webmasters/bin/answer.py?hl=en&answer=34439
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What are your opinions on the Google News vs Spanish Government Issue ?
Greg Sterling said: "Governments across Europe are justifiably alarmed by the declining fortunes of their respective newspaper industries. However punitive or parasitic taxation measures targeting Google, masquerading as copyright protections, are not the answer." Do you agree?
Industry News | | Tintanus1 -
Any sites out there that are better than freelancer and more local based and not international
Hey guys i am looking for someone local or in the us to handle my social media clients. ie: posts, blogging, tweets . Any sites where i can posts jobs better than freelancer. Also anyone know of sites that you can find bloggers to blog for your clients sites. ( more content writers , to take directions ) Thanks guys BizDetox
Industry News | | BizDetox0 -
Does Google still have a standard search result? How can I get it?
I have heard a lot from the experts that there are no "Standard" Google search results anymore. They said that most of the SERP's of Google that show up are customized/tailored for each individuals even if they are not logged-in using their Google Custom Search. My questions are, Is there still a way to retrieve the standard Google search result? How? Are these scripts will be helpful when searching on Google? *webhp?
Industry News | | RafaelRanada
*complete=0
*pws=0 watch?v=B8ofWFx525s B8ofWFx525s watch?v=B8ofWFx525s0 -
Are Wordpress sites being dinged by Google? Read a few articles regarding.
I read a couple "SEO" related articles that sites built in Wordpress are going to be dinged by Google because Google sees Wordpress sites as simple to make and a higher potential to be "spammy". Is there any truth to this? Your thoughts? I do give "thumbs up" and "best answer" marks and appreciate receiving thumbs up myself... Thanks
Industry News | | JChronicle1 -
Good time to start a niche site?
I am wondering, with all the recent changes in the algorithm, is this a good time to start a niche site? I know this can be fairly subjective, depending on the niche. But, anyone think this is a good opportunity?
Industry News | | inhouseseo0 -
Need a contractor to create Wikipedia pages
Hey guys! Can anyone recomend a good contractor to create/maintain Wikipedia pages? We are in the publishing business and I want to create Wikipedia pages for our authors/products. Need someone who has successfully created wikipedia pages before, can make basic research to find sources that wikipedia will consider reliable, has good academic writing skills and can start a debate in case they want to remove our article. If anyone knows good contractors please recommend. Thanks!
Industry News | | Alexey_mindvalley0 -
Google International and National Algorithm
Hi guys, I have a question. Do you have an experience about google national and international ranking algorithm. For example, is the same algorithm for " Google.de " Germany, and " Google.com "??? For example, lot of tactics, are valid and effective on google.de and not effective on google.com, tell us about this . Do you have any idea? Share your skills please! we need your help!
Industry News | | leadsprofi0