Is there a tool that I can use to scrape and see metatags?
-
Looking for a tool that allows me to scrape the websites off a page listing of Google and output a spreadsheet with the websites and their related meta-tag details (mainly title tag). Is there a tool out there that can conveniently allow me to do this?
-
Thanks for everyone's responses! Matt-Antonino, I'll look to implement that solution. Appreciate it!
-
Thanks for sharing Matt-Antonino.
That's good to know!
-
You can do this fairly easily with ImportXML in Google docs or xpath in Excel.
This is the guide that got me started:
https://www.distilled.net/blog/distilled/guide-to-google-docs-importxml/
If you use Chrome, finding the xpath is fairly simple and then you can just add it to your list.
The other way to do this is use Seer's Top 30 Results Scraper alongside the SEO Tools Excel plugin:
http://www.seerinteractive.com/blog/google-scraper-in-google-docs-update
In Excel tools simply use =htmltitle() and get the info you need.
-
Also keep in mind that search engines will change the title displayed in the SERP based on your query, so if you just get the info presented in the SERP, you may not be seeing the actual title tag.
-
Agreed Donna! It would be nice if Screaming Frog did that but it doesn't. I'm not sure of any existing tools that would do that.
You can run each site from the SERP through Screaming Frog to get all that data exported but that would take a good amount of time!
-
I don't think you can use Screaming Frog for that. It will certainly give you a ton of information about a single website, but it doesn't allow you to scrape Google search results and derive meta data for the top X results. Unless I'm missing something....
-
Screaming Frog is a great tool for this.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How happy should we be about a Page 1 “See results about” SERP?
A site fell off Page 1 organic listings but now regularly appears in the right-side “See results about“ SERP. How valuable are such results, especially compared to the 1-10 organic result listing? And, are “see results about” SERPs national?
On-Page Optimization | | PKI_Niles0 -
Is it safe to use Microformats and Schema.org on one page?
I'm running a news blog and I'm using schema.org/NewsArticle. I would like to know if I should add microformats on the article page too or just use the schema? If I add both, will it effect my sites SEO/ranking? Thank you
On-Page Optimization | | hakhan2010 -
Can horrific grammar and spelling in comments hurt the value of an otherwise great page?
I've got a website whose pages get lots of comments. Tons of activity, which I would think Google would like (and seems to like). However -- I just can't put this nicely -- most commenters are not very bright. Their grammar and spelling is horrific. These are not foreigners who lack English skills, they are just about all primarily English speakers and the site is 99% US traffic. It's a low-income segment of the population. So, I've been wondering recently if Google will mark down the value of the page due to the bad grammar and spelling in the comments, even if the page's content is otherwise very good and lengthy. I have read that they grammar and spelling into consideration when looking at the page, but would that include comments, or would they know they are comments and not judge a page on that? It would be a pain, but maybe I should I run all the comments at least through a spell checker? And manually fix their grammar? Problem is I get about 40 comments a day. And when I say bad grammar and spelling, I mean REALLY bad. Embarrassing.
On-Page Optimization | | bizzer0 -
On-page tool idea. What do you think? Like to hear it!
Easy on-page checker idea feedback Some script I wrote. Is this handy for someone? Try the script yourself like most things I like the Moz on-page grader (a lot!) but it does not do everything I want it to. So sometime I write my own little checks and other times I want to see list for of example all images without ALT tags. For this I wrote some code for myself the last few weeks. Now I bundled them for a handy little checker that is as is and not meant to make money with. It's just my handy work so to say and I would like to hear some feedback on the very simple but I think effective method of checking anchors on a front page against the page they link to. This is a very basis page checker for some important on-page ranking factors. If you type links in the option field then all links found on the page are listed Per link anchor quality check: The idea is very simple: if we assume for the simplicity of this method that the anchor text found first linking to an other page on the website this anchor text determnines the keyword with which the page is indexed. In made the links found on the page clickable. When clicked it starts the same scan but then only for the anchor text provided with the link clicked. If the on-page checks are ok I reccon your doing alright. idea is to start with the front page of any website and click the followed links, The page linked to is graded for match with anchor text as keyword. Try the script yourself Gr Daniel
On-Page Optimization | | DanielMulderNL0 -
Can I target one keyword with 2~3 pages?
Since my website is targeting a very specific field, there are not many widely searched keywords. So I'm thinking of targeting one keyword with 2~3 pages. 1. I've read Neil Patel's blog post on how to create dual rankings to make your search listings stand out on Google. So I assume it's okay to target one keyword with several pages. (http://www.quicksprout.com/2012/07/30/4-steps-to-making-your-search-listings-stand-out-on-google/ Step #2 Create Dual Rankings) 2. But I've also read things on Keyword Cannibalism saying that if you target one keyword with several pages, they will compete with each other, and Google will get confused. I'm wondering, is it okay to target one keyword with 2~3 pages? And is there any smart way to do it ? Thanks.
On-Page Optimization | | joony0 -
How would you target alternative names for your product with the use of internal/landing pages?
Recently, there has been some new industry names that have emerged for the type of software we provide and we want to make sure that we rank for those terms (of course!). The 3 names combined are too long for a title tag and would look odd to incl all 3 in the homepage. Any suggestions (or examples) of how this could done without looking spammy? I also don't really know where to put this within our website. e.g. enterprise portal, enterprise information portal (EIP) or corporate portal Any suggestions would be most welcome
On-Page Optimization | | alexpeps0 -
Can Rankings in Google differ so much from computer to computer.
I was telling my friend via facebook to go on my website, I told him to search 'nightlife forum' in google. To which, I believed it was 11th, top of second page. On his computer, its currently ranking at 1st place is it possible to have a difference of 10 places? even though he lives in the same city as me. Would be good to see what it ranks on your computers too google "nightlife forum" look for www.talknightlife.co.uk (don't get confused with the .COM one out there) Cheers Guys
On-Page Optimization | | Lukescotty0 -
SEF URLs. Should I use / or - ?
I have o activate SEF URLs in a website. Regarding SEO, is there any difference between using / or - ? I mean, Is it better to write URLs like this: http://www.domain.com/folder/folder/page or like this: http://www.domain.com/folder-folder-page ? Is there any difference? Thanks!
On-Page Optimization | | ociosu0