Measuring the size of a competitors website?
-
I think our website is too big, far too many indexed pages. I'd like to do some research on how big our competitors' websites are (how many indexed pages). Is there a way to do this?
Cheers,
Rhys
-
Thanks, both!
-
Xenu's Link Sleuth is free so you may want to check that out (not sure just now whether there are any limits with regard to website size) but I also recommend Screaming Frog - it's money well-spent, such a feature rich tool!
-
I highly recommend buying the license for Screaming Frog, at $100/year, you won't find a more valuable SEO tool for the money. You won't find a free (and trustworthy) that will crawl a site that large.
-
Hi Alick,
I tried that but I only have the free version so we're capped at 500 URLs. Also, the site:search provided 50,000 results, but I know we don't have that many pages. Are there any other tools?
Cheers,
Rhys
-
Hi,
Do a site: search on Google itself - like "site:google.com" - to return pages of SERPs containing the pages from your competitors site which Google has indexed.
I would also suggest you use screaming frog tool you will get more accurate value here.
Hope this helps.
Thanks
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
If few backlinks got deleted how they affect website?
Dear Experts,
Technical SEO | | jfdagborrbg
This is Julia and I am SEO Executive at J com. I would like to ask that If we are building 150+ quality backlinks per month for any website and 30 backlinks are deleting each month from 150 backlinks so It would be OK (natural) OR it would affect website ranking. I am looking forward to your response0 -
Entire website is duplicated on 2 domains - what to do?
My client's website has 1000+ pages and a Domain Authority of 23. I have just discovered that the entire site is duplicated on a second domain (main URL = companyname.com - duplicate site URL = company-name.com). The home page of the duplicate domain has a 301 redirect going to the main domain. However, none of the 1000+ other pages have any redirect set up, so Google is indexing the entire duplicate site. I'm assuming this is a bad thing for SEO. Duplicate site has a domain Authority of 4, so I'd like to transfer whatever link juice it has, towards the main site. What's the best thing to do? Ultimately I think it would be best to delete the duplicate site. So would it be a case of adding a redirect to the htaccess file along the lines of: redirect company-name.com/?slug? to https://companyname.com/?slug? (I realise this isn't the correct syntax - but is the concept correct?) Has anyone ever dealt with this successfully?
Technical SEO | | BottleGreenWebsites0 -
Website Revision
Hi all~ We are completely remaking our website: www.containmydog.com. I believe we have a good handle on the visual aspects of the redesign. What are the backend or behind the scenes (probably not using the technical term) things that need to be done so search engines know where things are. For example i know we are not going to remove some pages , change were some pages are on the site and add new pages. Is there a checklist that lists the important things to do when designing/redesigning a website? If there is not a checklist what are the things I should be asking the web person we hire?
Technical SEO | | PhotographerSteve1 -
Website not ranking but the blog is!
I am hoping someone might be able to help me, I am doing some work on a website. A new version of the site was recently launched and since then rankings have plummeted and the new blog pages are ranking better! When the new version of the site went live, the domain changed to the non-www version, plus an incorrect robots.txt file and we have never really been able to fully recover (both of these things were beyond my control!). The robots.txt file was corrected and some of the external links links changed to the non-www but there is a 301 redirect in place so changing to the non-www shouldn't have been the reason to drop the site out completely. Before the launch of the new website, the site was ranking on the front page of Google for a lot of relevant keywords such as outdoor blinds, outdoor blinds Perth, cafe blinds, patio blinds, etc. The quality of the links is pretty bad and I am attempting to remove them before doing a disavow of all the really bad quality links but unless we were really unlucky I don't think it's the links right now that are causing the problem. I have ran the site through numerous crawl tests, checked the robots.txt, there are no messages in GWMT, the pages are indexed but I have a feeling there is something wrong with the site that is stopping this site from ranking well. If anyone could give me any insights I would be really grateful. I know the site could be better structured from a keyword/ structure perspective but the site was ranking fine!
Technical SEO | | Karen_Dauncey0 -
Keep getting random broken links pointed at our website
Over the past 6 months I've noticed an increase of 404 errors being picked up by Google Webmaster Tools. When I look at the errors table and review these errors the links are invariable made up likes being generated from what looks like scraper sites or sites with search results embedded into them. Below are some examples of them. Broken Link URL Generating the Link www.mysite.com/rwickenhauser http://www.webstatsdomain.com
Technical SEO | | adamlcasey
www.mysite.com/guest http://www.webstatsdomain.com
www.mysite.com/windows http://www.webstatsdomain.com
www.mysite.com/..app http://213.174.143.40/
www.mysite.com/ht.. http://de.aguea.com/ None of these pages exist on our website.
So do I need to setup redirects for these or can they be ignored?0 -
Duplicate page issue in website
i found duplicate pages in my website. seomoz is showing duplicate web pages this is issue or not please tell me?
Technical SEO | | learningall0 -
How to SEO a Website Built off Godaddy?
I have a client whose website is built off Godaddy services. I know Godaddy is not the right choice for building a website, but what's done is done. The client has already bought the Godaddy services and there's no way I can tell him to go rebuild his website before we could optimize it for SEO. I'm already facing a lot of challenges while optimizing on-page elements. When I wanted to verify the ownership for Google Analytics and Webmaster Tool via his Godaddy account. the process failed many times. it looks like Godaddy is using some kind of caching not allowing us to modify the codes. For example, I'd applied the site verification codes for Webmasters Tool 48 hours ago, and the metatag for google site verification is not yet updated in the frontend. It's quite frustrating. What would you suggest?
Technical SEO | | suskanchan1 -
Removing pages from website
Hello all, I am fairly new to the SEOmoz community. But i am working for a company which organizes exhibitons, events and training in Holland. A lot of these events are only given ones ore twice and then we do not organise them any more because they are no longer relevant. Every event has its own few webpages which provide information about the event and are being indexed by Google. In the past we did not remove any of these events. I was looking in the CMS and saw a lot of events of 2008 and older which are being indexed. To clean the website and the CMS i am thinking of removing these pages of old events. The risk is that these pages have some links to them and are getting some traffic, so if i remove them there is a risk of losing traffic and rankings. What would be the wise thing to do? Make a folder with archive or something? Regards, Ruud
Technical SEO | | RuudHeijnen0