Tool for Generating Sitemap/ URL List
-
HI,
I'm looking for a tool that'll generate a URL list for a site. I looked at this thread here http://www.seomoz.org/q/online-sitemap-generator which came up when I searched for sitemap generator.
However, I don't need a sitemap per se, and I don't need to submit it to Google - just a list of pages is what I need.If it updated automatically, that would be useful as well.
Anyone know of a tool, on or offline? Or anyone used Xenu and know if it's what I'm looking for? Or is there a simple solution that I'm missing?
Thanks.
-
Try DYNO Mapper. If you are looking for a tool that will generate a URL list, I think it will work really well for you. It also will give you a full content inventory that you can filter, search, and export content types. This particular Sitemap Generator is also integrated with Google Analytics and has collaboration capability for multiple users.
-
Just spent the last couple hours checking out Xenu. Found this article here on SEOmoz:
http://www.seomoz.org/blog/xenu-link-sleuth-more-than-just-a-broken-links-finder.
Turns out Xenu is an awesome tool - can't wait to make more use of it.
I also took Joyce's advice which was more suited to the task I was trying to do. Thanks for the simple solution!
-
Yes Xenu will do it, it checks all the links and then offers you a sitemap as a report at the end
I like Xenu, it's quick too
-
If you're using SEOMoz Pro, you can enter the domain into Open Site Explorer and then click the Top Pages tab. This will give you an exportable list of up to 10,000 pages on the site. Alternatively, you can go to Google Analytics and click Content > Top Content. This will give you an even bigger list, though you may have to filter out any attributes that Google picks up.
Xenu can also be a good option, but you still have to do a lot of filtering.
I hope that helps!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Using keywords in my URL: Doing a redirect to /keyword
My website in "On Page Grade" received an A.Anyway, I only have 1 thing to optimize:_"Use Keywords in your URL__Using your targeted keywords in the URL string adds relevancy to your page for search engine rankings, assists potential visitors identify the topic of your page from the URL, and provides SEO value when used as the anchor text of referring links."_My website is ranking in top10 for a super high competitive keyword and all my others competitors have the keyword on their domain, but not for my URL.Since I can't change my domain for fixing this suggestion, I would like to know what do you think about doing a 301 redirect from / to mydomainname.com/keyword/So the index of my website would be the /keyword.I don't know if this can make a damage to my SERP for the big change ir it would be a great choice.
On-Page Optimization | | estebanseo0 -
Alt text / internal linking
Hi everyone A question about best practice when linking from pictures on our homepage - hirespace.com We have an option of using divs with background images (nicer in terms of design) but it means that we can't use anchor text or alt text to show Google what these internal links are about. The other option is to use images which do not allow us as much flexibility in terms of CSS but would allow us to use alt text. There is also an opinion that we should have separate text links at the bottom of the homepage to get the anchor page in. What is best practice in this situation - is alt text worth sacrificing some CSS flexibility for? How important is anchor/alt text for internal linking? Thanks guys.
On-Page Optimization | | HireSpace0 -
Redirect both / and non-/ URLs?
I am doing SEO on WP site. Due to some duplicate pages (rel canonical was done before) I am doing 301 redirects at the moment. And I wonder if I need to redirect both links w/ and w/o trailing slash. Default is non www, w/o trailing slash. Like there is .com/category/news but there is same page linked in .com/news (well it works when permalink structure is set to /%category%/%postname% and returns 404 error when structure is set to /%postname%).
On-Page Optimization | | OVJ
I redirected .lt/naujienos to .lt/category/naujienos. Should I also redirect .lt/naujienos/ (with trailing slash)? There's absolutely no problem redirecting this, but there are some more pages which I want to edit their URLs and I wonder If I should do both redirects from links /w and w/o slash?1 -
Articles URL
Hello, Currently, I am parsing article base on article ID on URL request. For example:
On-Page Optimization | | JohnHuynh
To go to the content of article **What is the visa on arrival? How to get it? **I am using URL like that http://www.vietnamvisacorp.com/news/what-is-the-visa-on-arrival--how-to-get-it--245.html and base on ID 245 to get this page content.
But, now I want to optimize this URL to http://www.vietnamvisacorp.com/news/what-is-the-visa-on-arrival--how-to-get-it.html. Then I got a problem: How to get an article without ID? Thanks,0 -
Which URL Structure is best for New Website?
Hello All, I have one client and he want to develop his website but the he want the URL structure for his website page like below: http://www.example.com/category/xxproduct.html But I have suggest him below URL Structure http://www.example.com/category/xx-product.html So Can you people please suggest me that from above which URL structure is better from SEO side as well as from Visual side..? Is using - is better to separate the words in URL?
On-Page Optimization | | jemindesai0 -
Tag-URLs in Magento
Hello, I have got a problem concerning Tag-URLs in Magento (the URLs mentioned are just fictitious 😞 At the moment, they look something like this: (1) http://store.com/tag/product/list/tagId/1/ ... so these URLs are not search engine friendly at all. Using a Magento extension you could transform them in speaking URLs: (2) http://store.com/tag/digital-cameras What would you do if you sold, say, digital cameras and your online shop ranked high for the keyword "digital camera" with URL No. 1 (not search engine friendly). Would you transform (1) in (2) and 301 all non speaking URLs? But would you keep the high ranking for "digital camera" when 301 to URL No. (2). But, what I'm most concerned of is : There is actually a landing page (category page) for the keyword "digital camera" : http://store.com/digital-cameras. Shouldn't the last URL rank high for "digital camera"? (instead of the tag URLs). But given the situation above, does it make sense now to 301 the tag URL to the category page? I would perhaps lose my good ranking, wouldn't I? Thanks a lot for your help! Martin
On-Page Optimization | | SmartyMarty810 -
URL extensions naming
I have always wrote URL extensions as www.mysite.com/two_words.html .... when I need to separate two words, I use _ as the separator ... I am a first time SEO Moz user ... I While looking around the tools on SEO Moz, I happened to stumble across the on-page analysis. A great tool indeed, rather worryingly though, one issue it flagged to me was my URL extension "Characters which are less commonly used in URLs may cause problems with accessibility, interpretation and ranking in search engines. It is considered a best practice to stick to standard URL structures to avoid potential problems." Can someone advice me if this really is a problem, its just not this project, its tons of sites I have already developed that I am also worried about ... I always write file extensions with more than one word using _ to separate the words. How should I write the extension, I am almost embarrassed to ask this question ... Surely, even Google's algorithms are not smart enough to decipher two words without some some sort of spacing .... Regards J
On-Page Optimization | | Johnny4B0 -
Trailing slash on URLs
Hi everyone My question is regarding trailing URLs in Wordpress A designer setup a site and made the URL structure something like: www.website.com/description/ but I want to change it to www.website.com/description as this is more SEF A trailing / was added to the permalink structure in Wordpress, if I change this will google see these as new URLs and will all the current URLs become 404? Or should I just leave it? Thanks in advance
On-Page Optimization | | webseoservices0