Is there a tool that I can use to scrape and see metatags?
-
Looking for a tool that allows me to scrape the websites off a page listing of Google and output a spreadsheet with the websites and their related meta-tag details (mainly title tag). Is there a tool out there that can conveniently allow me to do this?
-
Thanks for everyone's responses! Matt-Antonino, I'll look to implement that solution. Appreciate it!
-
Thanks for sharing Matt-Antonino.
That's good to know!
-
You can do this fairly easily with ImportXML in Google docs or xpath in Excel.
This is the guide that got me started:
https://www.distilled.net/blog/distilled/guide-to-google-docs-importxml/
If you use Chrome, finding the xpath is fairly simple and then you can just add it to your list.
The other way to do this is use Seer's Top 30 Results Scraper alongside the SEO Tools Excel plugin:
http://www.seerinteractive.com/blog/google-scraper-in-google-docs-update
In Excel tools simply use =htmltitle() and get the info you need.
-
Also keep in mind that search engines will change the title displayed in the SERP based on your query, so if you just get the info presented in the SERP, you may not be seeing the actual title tag.
-
Agreed Donna! It would be nice if Screaming Frog did that but it doesn't. I'm not sure of any existing tools that would do that.
You can run each site from the SERP through Screaming Frog to get all that data exported but that would take a good amount of time!
-
I don't think you can use Screaming Frog for that. It will certainly give you a ton of information about a single website, but it doesn't allow you to scrape Google search results and derive meta data for the top X results. Unless I'm missing something....
-
Screaming Frog is a great tool for this.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Using Bold text for keywords
Hello I am updating an old e-commerce website of mine and many keywords are in bold - shall I remove the bold tag or keep them there? This is for SEO.
On-Page Optimization | | xdunningx0 -
Should I leave away the http when using the canonical tag
My site redirects http://www.anydomainblabla.com to www.anydomainblabla.com So I guess I have to do it like this Is that correct? Big thanks for your time.
On-Page Optimization | | MichaelJanik0 -
Why am I not seeing sitelinks for this website?
http://laleablack.com/ It's been around for a few months - I have other sites showing sitelinks in the search results. but when I search for "laleablack" "lalea and black" I don't see anything...anyone have any idea why? Links have proper anchortext etc
On-Page Optimization | | smlsml0 -
Can PDFs on your website be picked up in search engines?
Can PDFs on your website be picked up in search engines? There's a ton of info online that says it can and can't be picked up.
On-Page Optimization | | JennH0 -
How about this "onpage overoptimisation" everybody is talking about? Are the on-page optimisation reports still to be used?
Are the on-page optimisation reports still to be used? If we do check all factors we risk penalization because of latest Panda update?
On-Page Optimization | | MugurCosminFrunzetti0 -
New sitelinks - can we control the number?
A quick question on Google's new sitelink format. When searching for our brand name (Confetti) Google returns 8 sitelinks for our site. When searching for our domain (confetti.co.uk) Google returns the maximum number of 12 sitelinks. Is there a quick way (Webmaster Tools for example) to increase the number of sitelinks for our brand name to 12? Thanks,
On-Page Optimization | | Confetti_Wedding0 -
Does Google still see masked domains as duplicate content?
Older reads state the domain forwarding or masking will create duplicate content but Google has evolved quite a bit and I'm wondering if that is still the case? Not suggesting that a 301 is not the proper way to redirect something but my question is: Does Google still see masked domains as duplicate content? Is there any viable use for domain masking other than for affiliates?
On-Page Optimization | | TracyWeb0 -
Can someone please help me identify where all these URLS to my homepage are coming from?
Hi. I installed the SEOmoz toolbar for Firefox, and analyzed my home page, then clicked on 'get a full site analysis at Site Explorer'. This is what came up: http://www.opensiteexplorer.org/www.frs-solutions.com%252Fcontent%252Fhome/a!links?src=mb I hope that link works. If not, the URL is www.frs-solutions.com Anyway, there are about 57 different URLS within my site all pointing to my homepage! I have no idea where they are coming from. Can someone with an experienced eye take a quick look and tell me what I might be up against? Thank you!
On-Page Optimization | | aprilm-1890400