Is there a tool that I can use to scrape and see metatags?
-
Looking for a tool that allows me to scrape the websites off a page listing of Google and output a spreadsheet with the websites and their related meta-tag details (mainly title tag). Is there a tool out there that can conveniently allow me to do this?
-
Thanks for everyone's responses! Matt-Antonino, I'll look to implement that solution. Appreciate it!
-
Thanks for sharing Matt-Antonino.
That's good to know!
-
You can do this fairly easily with ImportXML in Google docs or xpath in Excel.
This is the guide that got me started:
https://www.distilled.net/blog/distilled/guide-to-google-docs-importxml/
If you use Chrome, finding the xpath is fairly simple and then you can just add it to your list.
The other way to do this is use Seer's Top 30 Results Scraper alongside the SEO Tools Excel plugin:
http://www.seerinteractive.com/blog/google-scraper-in-google-docs-update
In Excel tools simply use =htmltitle() and get the info you need.
-
Also keep in mind that search engines will change the title displayed in the SERP based on your query, so if you just get the info presented in the SERP, you may not be seeing the actual title tag.
-
Agreed Donna! It would be nice if Screaming Frog did that but it doesn't. I'm not sure of any existing tools that would do that.
You can run each site from the SERP through Screaming Frog to get all that data exported but that would take a good amount of time!
-
I don't think you can use Screaming Frog for that. It will certainly give you a ton of information about a single website, but it doesn't allow you to scrape Google search results and derive meta data for the top X results. Unless I'm missing something....
-
Screaming Frog is a great tool for this.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Can you use the canonical tag and rel=next and rel=prev on category pages.
We have a conflict of information between our web developers and our SEO company. We are an on-line retail company hence we have a fair number of different categories. Our site is set up with the rel=next and rel=prev tags. Our SEO company have asked us to implement canonical links on our category pages and leave the rel=next and rel=prev tags as they are. Our web developers are saying by doing this we are asking Google to ignore all of our products on all of the pages except page 1 which would mean Google would not index a lot of our products. I have looked at a few articles but I am struggling to understand which way to go. Any advice would be appreciated. Thank you in advance.
On-Page Optimization | | Palmbourne0 -
Optimizing images on website. Is it bad to use the same alt names and keywords?
I have a webpage that I have 6 separate images showing step by step instructions of how to use our product with a sentence describing instruction inside image. I took screenshots of the instructions from the products App and uploaded them to webpage because they provide a great visual. I want to make sure I optimize correctly, can I use the same keywords for all the image names?
On-Page Optimization | | artscube.biz0 -
Trying to SEO a site that used Header Tags for Design
I am trying to SEO a website that was built years ago and uses Header tags for design. The site must have 25 and tags used for design purpose. Is there any way to work around this problem? Perhaps a code that tells Google to ignore these as Headers? The web designers say that they are looking to fix the problem sometime this summer but you never know if that means it a month away or years away. I really want to help this website but I believe that the Header tags are one of the reasons that his site does not show in the top 100 rankings for any keywords. Any help would be great. www.wallybuysell.com Chris.K
On-Page Optimization | | CKerr0 -
Can soft 404's hurt my rankings?
This post mainly pertains to soft 404's but I recently dropped a few ranks in my main keyword which I have maintained prior to this my better rank for well over 2 years. I participate in NO BLACKHAT and obtain links naturally. I want to describe a few issues that happened prior to my ranking dropping and see what you guys think. I started to receive about a week prior to my ranks dropping DNS issues with GWT. It was weird because when I would use Goole Fetch on those pages they would return just fine so I was not sure what was happening there. I use Google page speed server which did in fact decrease my load time (YEAH!!) so that was cool. About 1 week prior to my rankings dropping I enabled godaddy's Website Accelerator as well thinking that could help even more. Because of this I thought maybe this had something to do with my DNS issues with Google so I decided to turn off my website accelerator with Godaddy and just leave my Google pagespeed service on. I figure I don't need 2 of them anyways IMO. Also at the same time I started to receive a ton (31,000+) html errors with duplicate metadescriptions and titles. I discovered I had an error with my code which was displaying 2 different sets of descriptions and titles for each of these pages. I since then have fixed the issue and waiting for Google to index those pages. Here is were I think I might have been hurting from the drop in rankings. Some months ago (maybe 2) I decided to redirect my 404's to my homepage. Yes I know this is not good now and I have created a proper 404 page which returns the 404 code. I recently started getting a ton of Soft 404 errors in GWT which is what brought my attention to this issue. My question is, could my action of redirecting my users to my homepage as a 404 which obviously was returning a 200 on a page that did not exist be possibly the culprit to my ranks dropping?
On-Page Optimization | | cbielich1 -
How can I maintain my website's authroity if I change the domain name?
I have 2 sites. Site A is doing well and growing, but has a low Domain Authority. Site B is older, has a higher authority but I want to retire it as the site's topic is well dated. Can I move all of my content on site A, to site B, change site B's name to Site A's name keep its higher authority and add all of the content and 301 redirect from site A to site B? Whew, Thanks for any thoughts, Jeff Strassman
On-Page Optimization | | biggieshaws0 -
Can Your Site Get Penalized For Keyword Stuffing On An 'Untarged' Keyword?
My site has dropped since the EMD/Panda 20 roll out and I am looking for reasons why. I am looking at Keyword Stuffing as one potential problem. My web site is on the topic of WordPress Security with that being the main keyword I want to target. Now I can limit the number of occurrences of 'wordpress security' to below the recommended 15, but it is impossible to do this for 'wordpress' without severely compromising the user experience. I've got other content on topics such as WordPress Backup and WordPress Security Plugins etc, so obviously the word 'wordpress' is bound to appear frequently. Is there a risk that Google will penalize me for Keyword Stuffing on 'wordpress' and thus pull down the site or page for other keywords? Or would it simply mean I won't be able to rank for 'wordpress' (which I am quite happy about)? Thanks!
On-Page Optimization | | andersvin0 -
How can we get Google to offer postcard verification for our Place Page?
Most of the time, when we claim a Google Place Page, they give 2 choices to verify ownership: 1) phone verification and 2) postcard verification. But right now (and for several weeks), for our listing, they are only giving the phone verification choice, which unfortunately won't work with our automated phone system. How can we get our Place Page listing verified through a postcard sent to our address, when Google isn't presenting that as an option?
On-Page Optimization | | DenisL0