Measuring the size of a competitors website?
-
I think our website is too big, far too many indexed pages. I'd like to do some research on how big our competitors' websites are (how many indexed pages). Is there a way to do this?
Cheers,
Rhys
-
Thanks, both!
-
Xenu's Link Sleuth is free so you may want to check that out (not sure just now whether there are any limits with regard to website size) but I also recommend Screaming Frog - it's money well-spent, such a feature rich tool!
-
I highly recommend buying the license for Screaming Frog, at $100/year, you won't find a more valuable SEO tool for the money. You won't find a free (and trustworthy) that will crawl a site that large.
-
Hi Alick,
I tried that but I only have the free version so we're capped at 500 URLs. Also, the site:search provided 50,000 results, but I know we don't have that many pages. Are there any other tools?
Cheers,
Rhys
-
Hi,
Do a site: search on Google itself - like "site:google.com" - to return pages of SERPs containing the pages from your competitors site which Google has indexed.
I would also suggest you use screaming frog tool you will get more accurate value here.
Hope this helps.
Thanks
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Prioritizing SEO Items for a New Website Launch
Hello, a beginner SEO question I'd love to hear your perspective on. We're launching a new site and need to prioritize our SEO work. For a news site with multiple articles published daily, using RSS feeds to push stories to Google News is our top priority. Which item should we focus on next, an XML sitemap for Google, or an article directory based on our internal taxonomy that would provide backlinks for the individual posts to rank? Are XML sitemaps still relevant? Is an article directory a must-have or nice-to-have? Appreciate any suggestions you may have!
Technical SEO | | Gbtyne1 -
How preproduction website is getting indexed in Google.
Hi team, Can anybody please help me to find how my preproduction website and urls are getting indexed in Google.
Technical SEO | | nlogix0 -
How to handle pagination for a large website?
I am currently doing a site audit on a large website that just went through a redesign. When looking through their webmaster tools, they have about 3,000 duplicate Title Tags. This is due to the way their pagination is set up on their site. For example. domain.com/books-in-english?page=1 // domain.com/books-in-english?page=4 What is the best way to handle these? According to Google Webmaster Tools, a viable solution is to do nothing because Google is good at distinguishing these. That said, it seems like their could be a better solution to help prevent duplicate content issues. Any advice would be much welcomed. 🙂
Technical SEO | | J-Banz0 -
When do I change out my meta tags after a full website revamp?
We're creating a new version of our entire website - look and feel is completely different, though core functionality and results are the same. Just cleaner, faster.. etc. We're doing a temp redirect to the temporary url for testing and to slow roll the release to some of our users for a more friendly approach. Eventually, the new look and feel will be under the original url. I've researched best practices for the site transfer, including "make sure the meta tags for title and description are exactly the same". The concern I have is that Moz Analytics is detecting a lot of errors in the existing meta tags. They're too long, have changed and become inconsistent after being passed through different hands, & some have some keyword stuffing in there. I have plans to change them out and really clean them up... I'm just wondering, when is the best time to do that? Since the tags are bad, should I just do it now but make sure that the old and new are matching? Or should I wait (and for how long?) after the new site is switched over and everything is on the original URL?
Technical SEO | | SFMoz0 -
One hosting plan for multiple websites?
I use one Godaddy shared Linux hosting account for 4 separate websites. In Google Webamster Tools, specifally "Sitr Errors," I noticed that inner pages from another site are being listed as a broken link in the original unique-now-shared site. I checked and the files are not mi-installed. My question is, should each of the four sites have a unique hosting plan and/or static IP? Thanks, Eric
Technical SEO | | monthelie10 -
Reusing content owned by the client on websites for other locations?
Hello All! Newbie here, so I'm working through some of my questions 🙂 I do have two major question regarding duplicate content: _Say a medical hospital has 4 locations, and chooses to create 4 separate websites. Each website would have the same design, but different NAP, and contact info, etc. Essentially, we'd be looking at creating their own branded template. _ My question 1.) If the hospitals all offer similar services, with roughly the same nav, does it make sense to have multiple websites? I figure this makes the most sense in terms of optimizing for their differing locations. 2.) If the hospital owns the content on the first site, I'm assuming it is still necessary to change it duplicates for the other properties? Or is it possible to differentiate between the duplication of owned content from other instances of content duplication? Everyone has been fantastic here so far, looking forward to some feedback!
Technical SEO | | kbaltzell0 -
SIte cloned my entire website and is now outranking me
My site is http://www.medic8.com and http://www.mealldubh.org has cloned my site and is now outranking my site. I have submitted DMCA requests to Google with no response. I do not know what to do now but surely it should be obvoius that this site has cloned me and there must be a way to have this scraper removed from the index? Im lost in terms of what I can do next so any help would be greatly appreacited.
Technical SEO | | thefresh0 -
What is the largest page size a searchbot will crawl?
When setting up pagination, what should we limit the page size to? When will a searchbot stop crawling a particular page?
Technical SEO | | nicole.healthline0