Export Google Search Results
-
I would like to export a list of pages indexed by Google and other search engines. According to our site search with google, we have over 300,000 pages indexed. Anyway to export a list of all of these pages?
Tools like ScreamingFrog (which I own a copy of) can crawl your site but does not tell you the pages indexed by a search engine... I've tried using tools that will let you export each page of the results. However, this won't work for a 300K page website.
Thanks for your help!
-
Hi Shani! Do these responses help to answer your question or are you looking for more information? If you're good to go, please mark this as answered. Thanks!
-
Thank you Thomas and Logan! Thomas, GWT puts a cap at 999 pages, but it is also the search terms used to get to our site as opposed to the pages that are indexed. Logan, it appears your answer is accurate as I have been researching this for awhile with no luck.
-
There's no tool that will handle it in that kind of volume. I know of one tool called SERP Scraper that will do 100 URLs, but that's no good when you're evaluating 300k. I'm fairly certain Google makes it impossible for anyone to build this tool, as useful as it would be, it would exist already if this weren't the case.
-
I'm not aware of any tools that would be able to achieve what you need. However, would GWT not be able to help you here?
Search Analytics section would allow you to download all the pages that are indexed.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Skewed results in Google
Im just curious.. Im sure a lot of you, like me, Google your own search terms on a fairly frequent basis. Does doing this skew the result you are actually seeing? As in, does Google tailor the results more to my liking, and therefore perhaps show my own site (which I would click through to often enough) higher on the page that it actually is? Sometimes I swear that happens. Steve.
Search Behavior | | blitzna100 -
Google site: search showing twice amount of indexed pages. why?
I have around 50k pages indexed on my site but when I do a google site: of my site it shows around 100k pages indexed. Why is it showing so much more? It is also only showing around 700 pages indexed in my web masters account for the site. Background: We have a custom site map being generated automatically. Let me know if you would like more info, Thanks.
Search Behavior | | Nicktaylor10 -
Are there better & inexpensive third party website analytics software over Google Analytics?
I've heard there are some third-party software that provide greater depth of information than Google Analytics, such as mouse tracking, heat mapping, video snapshots ect. Can anyone recommend a good program to use? I've tried a basic web-search but there seems to be a great variety of different ones.
Search Behavior | | Justin_hannan270 -
Google analytics realtime reporting same keywords with different capitalization
I was just looking in my analytics and I saw something I have never seen before. Maybe its old news but its new to me. I have attached a screen shot. I have 2 keywords listed which are the same but are capitalized different scrabble dictionary Scrabble dictionary Does Google really consider these different? PlstfwR
Search Behavior | | cbielich0 -
Google Local
I am new to Google Local and just bagged a project. Can somebody guide me how to go about it. Is Google Local all about listing the business in Directory and Classified sites with complete address and profile? Any help will be highly appreciated.
Search Behavior | | KS__0 -
Google Places rankings go away, but organic rankings stay the same / get better?
Hello, I hope you all can help me out here with the information I could provide. Anyways, the past number of months I have had 1 of our clients rank extremely high organically and on Google places. For example: My client is a law firm so if you search “CLIENTS-TOWN Attorney” they would come up organically on the first page and if you were in that town and search “CLIENTS-TOWN Attorney” or even just “Attorney” they would also come up in the Google Places list. Now for some reason starting a week ago I noticed they have completely disappeared from Google Places. No changes have been done to their site and everything is the same as it was. Their organic searches are the same if not better than before. Do you guys have any clue as to why this happened and how I could possibly turn this around? Thanks for your help in advance!
Search Behavior | | WhiteHat120 -
Google Is Displaying An Alt image tag as my homepage's page title
Google is randomly displaying the alt image tag as the page title for my homepage. It happens when you search for the brand name, but the page title appears as "BrandName Logo" (obviously not "BrandName"). Has anybody seen this happen before?
Search Behavior | | MichaelWeisbaum0 -
Google Analytics Benchmarking Newsletter: How does your site perform?
With Google recently releasing benchmarking data I am curious as to what you all see across the various types of website niches that you work with (eCommerce, news, blog, services, small business, etc). And how SEO'd websites compare with this "raw" data provided by google. We have one medium size (12,000 products) strictly eCommerce website that has a bounce rate of 37% and an avg time on site of 5:20 While two other medium size eCommerce/blog sites have a bounce rate of 57% and 59% with average time on site of 2:37 and 2:30 respectively. Finally, I manage a website for a local small business that provides business and home cleaning services. This site has a bounce rate of 45% and 1:40 average time on site. How do your sites perform in these areas? Is it typical to see this great of a disparity between strict eCommerce websites and those sites that are both informational and transactional in nature? What about other kinds of websites? Cheers!
Search Behavior | | prima-2535091