How to create XML sitemap for larger website?
-
We need to create XML sitemap for a website that has more than 2 million pages. Please suggest me the best software to create XML sitemap for the website.
Since there are different strategies that larger websites submit sitemaps, let me know the best way to submit this sitemap for website of this size.
Or Is there any tool provided by SEOmoz for XML sitemap generation for larger websites?
-
Thanks a lot for your suggestion. We have successfully run 500 URI limit using trial version of this Screaming Frog software. We will check up with the licensed version of the software.
-
Our website is built with ASP and there is no plugin/extension for XML sitemaps
-
Screaming Frog is quite good I seem to remember, it will crawl all your URL's then create the sitemap for you. You may need to change the "priority" of the pages but the hard work is pretty much done for you.
-
What's the website built with? Doesn't it have a module/plugin/extension for XML sitemaps?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Sitemaps: Best Practice
What should and what shouldn't go in the sitemap? In particular, pages like subscribe to our newsletter/ unsubscribe to our newsletter? Is there really any benefit in highlighting those pages to the SEs? Thanks for any advice/ anecdotes 🙂
Intermediate & Advanced SEO | | Fubra0 -
Should I bother with a Video Sitemap?
Morning all, I've started a pretty aggressive Video content push in recent weeks. All our videos are on our YouTube channel. I decided to go with hosting the videos on YouTube based on my research on moz.com, especially considering the potential reach of the content on YouTube. What I'm finding is that the YouTube channel is doing great. We've hit 200 subscribers and 15K views in a little under a month. Wayyyy more than I could have ever hoped for. But the blog posts on our website are getting minimal traffic and no search visibility. That doesn't necessarily bother me, since the intention of our marketing campaign is to use YouTube to drive traffic to our website. So I guess my question is really more to do with optimizing the site with Video Sitemaps and best practices for Google Webmaster Tools. Right now we have YouTube videos embedded on blog posts like this one that have a time-stamp. But I've been working to create Gallery-style pages (no time-stamp) which would have multiple YouTube videos embedded on them like this one. These make it easier for visitors to watch multiple videos without needing to skip around to multiple blog posts. The challenge I'm running into is that when I go to submit a Video Sitemap to GWT I get an error saying that I have duplicate page content within the video sitemap. I've used several WP plugins to do this. It seems that when there is a video embedded on multiple URLs (pages + posts) the plugins will ignore the posts and only add the pages to the video sitemap. Here is my regular Sitemap Here is my video Sitemap I've attached a screenshot of my current Yoast Video SEO config if that's useful for reference. Does anyone have experience with using multiple sitemaps in GWT? I'm starting to think that maybe I shouldn't even bother with a video sitemap. Maybe those gallery-style pages should just go in the regular sitemap? Any thoughts or advice would be highly appreciated! Thanks llQfydA
Intermediate & Advanced SEO | | TMHoward860 -
Need some help/input about my Joomla sitemap created by XMap
Here is my current sitemap for my site http://www.yakangler.com/index.php?option=com_xmap&view=xml&tmpl=component&id=1 I have some questions about it's current settings. I have a component called JReviews that xmap produces a separate link for each category. ex: http://www.yakangler.com/fishing-kayak-review/265-2013-hobie-mirage-adventure-island 2014-09-03T20:46:25Z monthly 0.4 http://www.yakangler.com/fishing-kayak-review/266-2012-wilderness-systems-tarpon-140 2014-06-03T15:49:00Z monthly 0.4
Intermediate & Advanced SEO | | mr_w
http://www.yakangler.com/fishing-kayak-review/343-wilderness-systems-tarpon-120-ultralite 2013-11-25T06:39:05Z monthly 0.4 Where as my other articles are only linked by the content category. ex: http://www.yakangler.com/news monthly 0.4
http://www.yakangler.com/tournaments monthly 0.4
http://www.yakangler.com/kayak-events monthly 0.4
http://www.yakangler.com/spotlight monthly 0.4 Which option is better?0 -
Do I redirect pages that no longer appear on the website?
Here is an example of the link that is no longer on the website (Broken link) http://www.weddingrings.com/item.cfm?str_shortdesc=UNIQUE The broken link was fixed to : http://www.weddingrings.com/item.cfm?str_shortdesc=UNIQUE CARRE CUT DIAMOND ETERNITY BAND&str_category=Diamond-Bands-and-Gold-Rings&grouping_id=9&category_id=21&int_item_id=6884 Would I still need to redirect the old broken link to the new fixed one using 301 redirect?
Intermediate & Advanced SEO | | alexkatalkin0 -
Moving Content To Another Website With No Redirect?
I've got a website that has lots of valuable content and tools but it's been hit too hard by both Panda and Penguin. I came to the conclusion that I'd be better off with a new website as this one is going to hell no matter how much time and money I put in it. Had I started a new website the first time it got hit by Penguin, I'd be profitable today. I'd like to move some of that content to this other domain but I don't want to do 301 redirects as I don't want to pass bad link juice. I know I'll lose all links and visitors to the original website but I don't care. My only concern is duplicate content. I was thinking of setting the pages to noindex on the original website and wait until they don't appear in Google's index. Then I'd move them over to the new domain to be indexed again. Do you see any problem with this? Should I rewrite everything instead? I hate spinning content...!
Intermediate & Advanced SEO | | sbrault741 -
Is there a way to keep sitemap.xml files from getting indexed?
Wow, I should know the answer to this question. Sitemap.xml files have to be accessible to the bots for indexing they can't be disallowed in robots.txt and can't block the folder at the server level. So how can you allow the bots to crawl these xml pages but have them not show up in google's index when doing a site: command search, or is that even possible? Hmmm
Intermediate & Advanced SEO | | irvingw0 -
Website change of address
Hi Everyone, I apologize if the answer to this questions is obvious, but I wanted some input on how changing our web address of our site will affect our SERP. We are looking to change our website address from a.com to b.com due to rebranding of our company (primarly to expand our product line as our current url and company name are restricting). I understand that this can be done using 301 direct and via webmaster tools with google. My question is how does this work exactly? Will our old website address show in SERP rankings, and when a user clicks on the listing are they redirected to our new address? With regards to building new links from press releases etc, do we have links point to our new web address or the old one in order to increase SERP? Does google see our old address and new address as the same website and therefor it does not matter where inbound links point to and both will increase our ranking positions? It took 6 years of in house seo to get our website to rank on the first page of all the major search engines for our keywords, so we am being very cautious before we do anything. Thanks everyone for your input, it is greatly appreciated 🙂
Intermediate & Advanced SEO | | AgentMonkey0 -
Are there any SEO Tips before killing a website?
Hey guys, My company acquired another company, and after a couple of months we decided to completely kill their website. I'm not finding any info about SEO best practices for this type of situation. From the "switching domains" and "new sites" articles and blog posts I can extrapolate that I should: 301 redirect their home page to ours Look at specific pages with good authority that relate to our pages and 301 them. Look at the strongest backlinks to their site and try to change them to point to our site. Create a 404 page for the rest of their webpages that tells them that we acquired the company (hopefully with a main menu and search bar) Any other suggestions?
Intermediate & Advanced SEO | | nrv0