Can SEO Moz perform a full site crawl and provide a report showing all URLs within an existing domain?
-
We are conducting a site redesign and need to get an idea of all pages that are out there on our domain (in some report fashion). This would help for discovery and cleanup as we re-work the site and move to a new CMS.
Thanks
-
Hey Stephen,
THanks - I have heard of Xenu many times before. I'll try that out.
I appreciate the input!
-
just going to throw in an alternative for this type of thing Xenu Link sleuth - free, easy and powerful.
Yes that site looks like ass but the tool is great. Dr Pete and others have done "how to use" posts on seomoz - just search xenu on seomoz
-
Yes - that's exactly it. Can I just do an export of the campaign crawl? or is that found somewhere else? Forgive my ignorance...still new to the tool and just can't seem to find.
-
Are you looking at your crawl report? It is available in an Excel file which provides the exact information you desire.
-
Hey Ryan,
Thanks for the response.
It's not a huge site. I would say there's probably around 600 total pages. It definitely looks like the SEO crawler has crawled the entire site, but I can't seem to find a way to just get a one-shot view of all the pages it's accessed.
Any ideas on where this can be found?
Thanks again!
-
How many pages are on your site?
The SEOmoz crawler will do exactly what you want but there is a URL limit. You should also understand it will experience the same limitations of any crawler. It wont list pages which behind a user log-in nor pages which are blocked by robots.txt. I am pretty sure the SEOmoz crawler doesn't list secure pages either but last I heard they were working on including them.
If your site has any island pages they wont be discovered either.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Moving pages to new domain
Hello, Our product pages are ranked #1 on google for our target keywords using our domain e.g. www.olddomain.com/cases/productxyz and sell about 20 products all ranked #1. We have a new company called www.newco.com/case/product1, 2, 3 etc. We use woocommerce e-commerce for both old and new sites. What is the best way to list our old co-products on our new site and move over the #1 rankings? Do we create new products (using our new nice design) in the newco.com woo commerce and then redirect old co links? do we copy and paste all that old content into the newco.com? Totally confused. Thank you!
Web Design | | Jamesmcd031 -
Conversion drop after site move
Have just migrated current and active site to Magento 1.9 and a new hosting package (dedicated and reasonably powerful for what we need). But we seem to have had a drop in conversion rate. I don't really know whether it's anything to be concerned about the moment. I've set up extensive server level 301 redirects for old pages. Would be glad of any thoughts? Thanks you
Web Design | | seoman100 -
New ecommerce site: Close old site and full domain redirect or keep it linking to new site?
We have rebranded and are working on our new site (B). Our old site (A) has a much higher domain/page authority than our new site. Currently we have the original Site A still there, with all links/pages pointing to the new Site B when people click. I am unsure whether we'd be best to close down the Site A completely and do a full domain redirect to Site B. Site A: 10 years age and has a moderate amount of links to it.
Web Design | | ModowestNZ
Homepage - PA: 24 DA:11 Site B: 6 months age, few links
Homepage - PA: 1 DA:2 My concern with the full domain redirect is that the indexed/ranking pages would dissapear. The benefit is less brand confusion for our niche range of party accessories.0 -
Is it Bad to Break Up A Site into Multiple Sites?
I have a big cluttered website with endless pages. It's a non-profit that has content for patients, researchers, therapists, etc.. Would it be a bad idea to turn this cluttered site into 3 or more completely different sites, each focused on their specific demographic? Or should I just figure out how to organize the one site better? Thanks for your help!!!
Web Design | | bosleypalmer0 -
How do I reduce my vast number of errors that SEO Moz has picked up?
Hi, The number of errors on my site recently jumped up by 450. Most of them are duplicate page content and title errors. I am confused as to how the site suddenly got so many errors. I recently added a new sitemap extension to the site as I was told by my developers that it was hacked a couple of months ago. Please could someone help resolve these errors?
Web Design | | Saunders18650 -
Thoughts on how to speed up my site? (Other site ideas.)
I was wondering if someone had a few minutes to talk a look at our site and see how to speed it up. I've ran some of the speed tests and I get different results, so I'm not sure if my site would be considered "slow" or not. Also if there any other things that jump out at you (usability, side bar, things I should change / add / take away to make a users experience better, etc.) please let me know. Any feedback is good feedback to me. 🙂 I'm using WP Super Cache as well. Thanks!
Web Design | | NoahsDad1 -
Infinite scrolling - is it SEO friendly ?
If i am trying to implement infinite scrolling and remove pagination completely, will it effect my SEO ? Why or why should not infinite scrolling be implemented ?
Web Design | | Myntra0