Tool for Generating Sitemap/ URL List
-
HI,
I'm looking for a tool that'll generate a URL list for a site. I looked at this thread here http://www.seomoz.org/q/online-sitemap-generator which came up when I searched for sitemap generator.
However, I don't need a sitemap per se, and I don't need to submit it to Google - just a list of pages is what I need.If it updated automatically, that would be useful as well.
Anyone know of a tool, on or offline? Or anyone used Xenu and know if it's what I'm looking for? Or is there a simple solution that I'm missing?
Thanks.
-
Try DYNO Mapper. If you are looking for a tool that will generate a URL list, I think it will work really well for you. It also will give you a full content inventory that you can filter, search, and export content types. This particular Sitemap Generator is also integrated with Google Analytics and has collaboration capability for multiple users.
-
Just spent the last couple hours checking out Xenu. Found this article here on SEOmoz:
http://www.seomoz.org/blog/xenu-link-sleuth-more-than-just-a-broken-links-finder.
Turns out Xenu is an awesome tool - can't wait to make more use of it.
I also took Joyce's advice which was more suited to the task I was trying to do. Thanks for the simple solution!
-
Yes Xenu will do it, it checks all the links and then offers you a sitemap as a report at the end
I like Xenu, it's quick too
-
If you're using SEOMoz Pro, you can enter the domain into Open Site Explorer and then click the Top Pages tab. This will give you an exportable list of up to 10,000 pages on the site. Alternatively, you can go to Google Analytics and click Content > Top Content. This will give you an even bigger list, though you may have to filter out any attributes that Google picks up.
Xenu can also be a good option, but you still have to do a lot of filtering.
I hope that helps!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Is the URL Matching the Page Title Important?
Hello I have tried searching for an answer on this but I can't get a clear answer due to the results when searching for URL title. I have just launched our second Shopify site for one of our brands. My first site launched in 2014 but when I launched I didn't pay much heed to SEO for page titles, URLs, etc so have retrospectively fixed this over time. For my Shopify site just launching I want to get it as right as possible from the start (learning from mistakes). My question is regarding URLs and what my approach should be for better SEO. So, I have a page with a Title of Newton Leather Wallets, Purses, Card Holders & Glasses Cases and the URL is https://www.tumbleandhide.com/collections/newton-leather-wallets-card-holders It was my understanding that I should try and make the URL reflect the Page Title more accurately. The problem is that this takes the character count to 77. On other pages it can be in the 80s. Will the above link be better for SEO than say just https://www.tumbleandhide.com/collections/newton I am just wary of the URL's being too long as my Moz Site Crawl is returning a lot of URLs that are too long. Thanks in Advance.
On-Page Optimization | | lukegj0 -
SEO results/title tags/desktop vs. mobile
I am trying to figure out why my title tags comes up different between desktop and mobile search results. Desktop returns my title tag as written, but on mobile I get something completely different. It's related to the site, but not anything I can read, as coded in the site (i.e. not the title tags, meta, or anywhere else). Has anyone else experienced this? My title tag is 64 characters - I know it's a little bit over, but would that cause such a weird issue as a completely different title in the search results?
On-Page Optimization | | tallyhodesign0 -
URL / Meta info for Author bio pages
When you create a biography page for authors that point back to their G+ account, do you include any other signals in the URL other than the authors name? For example, would you use www.domain.com/sam-spade or www.domain.com/sam-spade-biography, etc? Similar question for the meta description. Do you add any signals there? Or, is rel=author strong enough. Best,
On-Page Optimization | | ChristopherGlaeser
Christopher0 -
Too many on page links in sitemap.html
My crawl report is flagging an issue with too many links to one of my pages, this page is my sitemap.html. However, I have coded the page so that if required is specified it generates an .xml version of the page and if not then the html version is displayed. What is the best way to stop the crawl finding the html version whilst maintaining it on the site for clients navigation?
On-Page Optimization | | SamPenno0 -
State Abbreviations and/or Fully Spelled Out?
Should State names be abbreviated and/or fully spelled out in title tags, meta descriptions and body content? Does Google know that VT = Vermont?
On-Page Optimization | | lbohen0 -
Ecommerce Product Subcategory URL
Our website has 5 main categories displayed in tabs in the header. The main landing page of each of the 5 categories is a paginated page (3pages- set up with canonical tags to avoid duplicate content) with a side bar which splits the main category into many subcategories. Each of these subcategories essentially filter the main landing page into more defined categories customers find useful (price/colour) BUT once clicked enter into a separate landing page. We have worked hard to avoid any duplicate content issues between these sub-landing pages and the main landing page. This was done as we wanted each of the subpages to organically rank (thus we went with this method rather than filters). Hope we didn't do the wrong thing there? The question is should these sub-landing pages route straight from home to have the best chance to get individually ranked or routed through the main category bearing in mind we have 5 main categories each with many subcategories. i.e. domain.co.uk/subcategory or domain.co.uk/category/subcategory Thanks in advance for any advice given.
On-Page Optimization | | jannkuzel0 -
How do you see a list of URLs with duplicate page titles?
When looking at the Duplicate Page Title report, the Other URLs column has various numbers that presumably indicate the number of pages that share the same title. When I click on one of these numbers, say a URL that shows 4 in that column, the next page reports "No sample duplicate URLs to report". Why isn't it showing me the other 3 URLs with the same page title?
On-Page Optimization | | jkenyon0 -
Site URL's
We are redeveloping our website, and have the option to amend URLs (with 301 redirects from old URL to new), so my question is: Would 'golfsite.com/golf-clubs' achieve superior rankings than 'golfsite.com/clubs' for the search term 'golf clubs' if all other factors were the same? Should the URL reflect the intended search term wherever possible?
On-Page Optimization | | swgolf1230