Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
Should 301-ed links be removed from sitemap?
-
In an effort to do some housekeeping on our site we are wanting to change the URL format for a couple thousand links on our site. Those links will all been 301 redirected to corresponding links in the new URL format. For example, old URL format: /tag/flowers as well as search/flowerswill be 301-ed to, new URL format: /content/flowers**Question:**Since the old links also exist in our sitemap, should we add the new links to our sitemap in addition to the old links, or replace the old links with new ones in our sitemap? Just want to make sure we don’t lose the ranking we currently have for the old links.Any help would be appreciated. Thanks!
-
I'm going to disagree a little bit with the other commenters. I've done quite a few large scale redirect projects and I'm not 100% opposed to using a "dirty sitemap" for a short duration. The better option is to leave some internal links pointed at the old URLs. I know what the search engines say, but I also know what I've experienced when it comes to getting 301'd links crawled again.
Read this post by Everett Sizemore for more info at what I'm describing:
http://moz.com/blog/uncrawled-301s-a-quick-fix-for-when-relaunches-go-too-well
-
"A sitemap should only contain links to active pages."
Hi shawn81
Alex is absolutely correct there.
In fact, Duane Forrester has said repeatedly that Bing absolutely does not like to find such pages in a sitemap and that you should make sure there are never 3XX, 4XX or 5XX status pages included because it will stop Bingbot from crawling your site.
While Googlebot is not so sensitive, the reality is that all search engines allocate a certain amount of crawl capacity for your site...if your sitemaps include a load of pages that are not likely to be indexed, the result is twofold:
- you are wasting capacity on useless pages and the crawler may never get to the stuff you really want indexed
- if the crawler encounters a lot of non-active pages when it crawls, future crawl capacity (not to mention trust) is likely to be reduced
Replace the old URLs with the new and give the bots a little thrill of adventure
Hope that helps,
Sha
- you are wasting capacity on useless pages and the crawler may never get to the stuff you really want indexed
-
There shouldn't be any 301 links in a sitemap. A sitemap should only contain links to active pages. So in your case, you should remove all the 301 links and replace them with the new links.
Couple notes - Having 301 links in your sitemap won't hurt your site or SEO unless the sitemap is so huge that you need to split it up into multiple files. But you should really only have the final links in the sitemap, neither people nor bots want to be redirected around. If you properly 301'd the crawlers will automatically update their links.
Changing links around in the sitemap generally won't hurt your site. Especially if the links no longer exist and you're improving the list. There are very few cases where making changes will hurt the site.
-
We have had a problem with this ourselves. We put a 301 redirect on our domain when we were building a new site (went from new. to www.) and search engines are still crawling the new. domain. Bing webmaster tools registers it as an error because they can't find the old site. I would lean toward removing it just because your users are probably being redirected somewhere they wouldn't necessarily want to go.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
If I'm using a compressed sitemap (sitemap.xml.gz) that's the URL that gets submitted to webmaster tools, correct?
I just want to verify that if a compressed sitemap file is being used, then the URL that gets submitted to Google, Bing, etc and the URL that's used in the robots.txt indicates that it's a compressed file. For example, "sitemap.xml.gz" -- thanks!
Technical SEO | | jgresalfi0 -
Image Sitemap
I currently use a program to create our sitemap (xml). It doesn't offer creating an mage sitemaps. Can someone suggest a program that would create an image sitemap? Thanks.
Technical SEO | | Kdruckenbrod0 -
301 with nofollow ?
Hi, our ecommerce link penalty was revoked by google back in Feb 26th 2013, but to this day we have not seen any improvement on our rankings. Due to 80% revenue loss we had to layoff quite a few people to stay alive. Situation now is more dire then ever for our company. We have millions of dollars invested in our business and google just busted it for some "low quality" or "spammy links" as they call it. We want to try to move to a different domain and do a 301 from the old domain to make sure our previous customers can still find us as a last effort to stay alive. But doing so we do not want to the bad links juice to flow to our new domain. Can we do a 301 with nofollow and will that have any negative impact or any impact at all.? any suggestion is greatly appreciated. Thank you Nick We are planning on moving to a different domain after 10 years, and laying off bunch of people due to loss of revenue.
Technical SEO | | orion680 -
Remove html file extension and 301 redirects
Hi Recently I ask for some work done on my website from a company, but I am not sure what they've done is right.
Technical SEO | | ulefos
What I wanted was html file extensions to be removed like
/ash-logs.html to /ash-logs
also the index.html to www.timports.co.uk
I have done a crawl diagnostics and have duplicate page content and 32 page title duplicates. This is so doing my head in please help This is what is in the .htaccess file <ifmodule pagespeed_module="">ModPagespeed on
ModPagespeedEnableFilters extend_cache,combine_css, collapse_whitespace,move_css_to_head, remove_comments</ifmodule> <ifmodule mod_headers.c="">Header set Connection keep-alive</ifmodule> <ifmodule mod_rewrite.c="">Options +FollowSymLinks -MultiViews</ifmodule> DirectoryIndex index.html RewriteEngine On
# Rewrite valid requests on .html files RewriteCond %{REQUEST_FILENAME}.html -f RewriteRule ^ %{REQUEST_URI}.html?rw=1 [L,QSA]
# Return 404 on direct requests against .html files RewriteCond %{REQUEST_URI} .html$
RewriteCond %{QUERY_STRING} !rw=1 [NC]
RewriteRule ^ - [R=404] AddCharset UTF-8 .html # <filesmatch “.(js|css|html|htm|php|xml|swf|flv|ashx)$”="">#SetOutputFilter DEFLATE #</filesmatch> <ifmodule mod_expires.c="">ExpiresActive On
ExpiresByType image/gif "access plus 1 years"
ExpiresByType image/jpeg "access plus 1 years"
ExpiresByType image/png "access plus 1 years"
ExpiresByType image/x-icon "access plus 1 years"
ExpiresByType image/jpg "access plus 1 years"
ExpiresByType text/css "access 1 years"
ExpiresByType text/x-javascript "access 1 years"
ExpiresByType application/javascript "access 1 years"
ExpiresByType image/x-icon "access 1 years"</ifmodule> <files 403.shtml="">order allow,deny allow from all</files> redirect 301 /PRODUCTS http://www.timports.co.uk/kiln-dried-logs
redirect 301 /kindling_firewood.html http://www.timports.co.uk/kindling-firewood.html
redirect 301 /about_us.html http://www.timports.co.uk/about-us.html
redirect 301 /log_delivery.html http://www.timports.co.uk/log-delivery.html redirect 301 /oak_boards_delivery.html http://www.timports.co.uk/oak-boards-delivery.html
redirect 301 /un_edged_oak_boards.html http://www.timports.co.uk/un-edged-oak-boards.html
redirect 301 /wholesale_logs.html http://www.timports.co.uk/wholesale-logs.html redirect 301 /privacy_policy.html http://www.timports.co.uk/privacy-policy.html redirect 301 /payment_failed.html http://www.timports.co.uk/payment-failed.html redirect 301 /payment_info.html http://www.timports.co.uk/payment-info.html1 -
Should I include tags in sitemap?
Hello All, I was wondering if you should include tags and categories in your sitemap. In the past on previous blogs I have always left tags and categories out. The reason for this is a good friend of mine who has been doing SEO for a long time and inhouse always told me that this would result in duplicate content. I thought that it would be a great idea to get some input from the SEOmoz community as this obviously has a big affect on your blog and the number of pages indexed. Any help would be great. Thanks, Luke Hutchinson.
Technical SEO | | LukeHutchinson1 -
Host sitemaps on S3?
Hey guys, I run a dynamic web service and I will start building static sitemaps for it pretty soon. The fact that my app lives in a multitude of servers doesn't make it easy to distribute frequently updated static files throughout the servers. My idea was to host the files in AWS S3 and point my robots.txt sitemap directive there. I'll use a sitemap index so, every other sitemap will be hosted on S3 as well. I could dynamically mirror the content from the files in S3 through my app, but that would be a little more resource intensive than just serving the static files from a common place. Any ideas? Thanks!
Technical SEO | | tanlup0 -
How to generate a visual sitemap using sitemap.xml
Are there any tools (online preferably) which will take a sitemap.xml file and generate a visual site map? Seems like an obvious thing to do, but can't find any simple tools for this?
Technical SEO | | k3nn3dy30 -
How to remove the 4XX Client error,Too many links in a single page Warning and Cannonical Notices.
Firstly,I am getting around 12 Errors in the category 4xx Client error. The description says that this is either bad or a broken link.How can I repair this ? Secondly, I am getting lots of warnings related to too many page links of a single page.I want to know how to tackle this ? Finally, I don't understand the basics of Cannonical notices.I have around 12 notices of this kind which I want to remove too. Please help me out in this regard. Thank you beforehand. Amit Ganguly http://aamthoughts.blogspot.com - Sustainable Sphere
Technical SEO | | amit.ganguly0