Sitemaps during a migration - which is the best way of dealing with them?
-
Many SEOs I know simply upload the new sitemap once the new site is launched - some keep the old site's URLs on the new sitemap (for a while) to facilitate the migration - others upload both the old and the new website together, to support the migration. Which is the best way to proceed? Thanks, Luke
-
Very much appreciated CleverPhD!
-
Found this while looking for a answer for another question could not find this the other day- right from the mouth of Google to not include pages that do not exist in XML sitemaps.
http://googlewebmastercentral.blogspot.com/2014/10/best-practices-for-xml-sitemaps-rssatom.html
URLs
URLs in XML sitemaps and RSS/Atom feeds should adhere to the following guidelines:
- Only include URLs that can be fetched by Googlebot. A common mistake is including URLs disallowed by robots.txt — which cannot be fetched by Googlebot, or including URLs of pages that don't exist.
-
Mate nailed it completely!
-
I would say make sure that your new sitemap has all the latest URLs. The reason people say that you should have old URLs in the sitemap is so that Google can quickly crawl the old URLs to find the 301s to the new URLs.
I am not convinced that this helps. Why?
Google already has all your old URLs in its systems. You would be shocked how far back Google has data on your site with old URLs. I have a site that is over 10 years old and I still see URL structures referenced in Google from 7 years ago that have a 301 in place. Why is this?
Google will assume that, "Well, I know that this URL is a 301 or 404, but I am going to crawl it every once in a while just to make sure the webmaster did not do this by mistake." You can notice this in Search Console error or link reports when you setup 301s or 404s, they may stay in there for months and even come back once they fall out of the error list. I had an occurrence where I had some old URLs showing up in the SERPs and various Search Console reports for a site for 2 years following proper 301 setups. Why was this happening?
This is a large site and we still had some old content still linking to the old URLs. The solution was to delete the links in that old content and setup a canonical to self on all the pages to help give a definitive directive to Google. Google then finally replaced the old URLs with the new URLs in the SERPs and in the Search Console reports. The point here being that previously our site was giving signals (links) that told Google that some of the old URLs were still valid and Google was giving us the benefit of the doubt.
If you want to have the new URLs seen by Google, show them in your sitemap. Google already has all the old URLs and will check them and find the 301s and fix everything. I would also recommend the canonical to self on the new pages. Don't give any signals to Google that your old URLs are still valid by linking to them in any way, especially your sitemap. I would even go so far as to reach out to any important sites that link to old URLs to ask for an updated link to your site.
As I mentioned above, I do not think there is an "advantage" of getting the new URLs indexed quicker by putting old URLs in the sitemap that 301 to the new URLs. Just watch your Google Search Console crawl stats. Once you do a major overhaul, you will see Google really crawl your site like crazy and they will update things pretty quick. Putting the old URLs in the sitemap is a conflicting signal in that process and has the potential to slow Google down IMHO.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Sitemap - What are the recommendations on the number of links
Hi, I have a sitemap(s) which is very large(.i.e. 60000) links, is it recommended to have so many links and how come when I do a site search(site:mydomain) the number of links are less than on my site map?
Intermediate & Advanced SEO | | FreddyKgapza0 -
Spotify XML Sitemap
All, Working on an SEO work up for a Spotify site. Looks like they are using a sitemap that links to additional pages. A problem, none of the links are actually linked within the sitemap. This feels like a strong error. https://lubricitylabs.com/sitemap.xml Thoughts?
Intermediate & Advanced SEO | | dmaher0 -
Changing sitemaps in console
Hi there, Does anyone have any experience submitting a completely new sitemap structure - including URLs - to google console? We've changed our sitemap plug in, so rather than /sitemap-index.xml, our main sitemap home is /sitemap.xml (as an example). Is it better to 410 the old ones or 301 redirect them to the new sitemaps? If 301, what do we do about sitemaps that don't completely correlate - what was divided into item1.xml, item2.xml is now by date so items-from-2015.xml, items-from-2016.xml and so on. On a related note, am I right in thinking that there's no longer a "delete/ remove sitemap" option on console? In which case, what happens to the old ones which will now 404? Thanks anyone for any insight you may have 🙂
Intermediate & Advanced SEO | | Fubra0 -
Legal Client Wants to Change Domain Name... What's the best way to pass authority from old domain?
Hey Mozzers,
Intermediate & Advanced SEO | | WhiteboardCreations
I received a call on Friday from a 6 attorney law office who have been my client for a long time. They have an established brand/domain in their market which isn't very big, but has a lot of competition. 2 of the attorneys are leaving to start their own firm and they want to remove a letter from their name abbreviation, thus their domain name as well. So, the other partners want to change the domain to reflect this. They want to buy a EMD [city]lawyers.com for about $1,600 along with some others to protect their new brand and name. I have a good idea as to what I need to do, BUT would love to hear advice from the community for this type of drastic change. 301 redirects? New Google Analytics code or same just different profile? Webmasters verifications? Content from old site? Old domain forwarding or keep active for a little bit? Is not the time to get them an SSL? Also, what should I prepare them for in terms of website traffic expectations and Google authority drops or remains the same? I know their Moz DA/PA will drop to 1/1, but anything else to look out for? Thank you in advance!
Fellow Pro Member - Patrick1 -
The best tool
Hi friends !! I have a huge question . Which is the best tool for SEO? I am using a lot of tools but I would like to know more ways to position my website in the top . I hope that you can help me! Regards , Carlos Zambrana
Intermediate & Advanced SEO | | CarlosZambrana1 -
Where to learn how best to promote content?
So now I created some really good content (with help of Egol and Peter here on moz.com) and now I need to promote it. To get it in front of authoritative sites so they hopefully will write about and link to it. I erroneously figured it would be fairly easy. I contacted two writers of a high level industry blog/magazine that previously had mentioned us in press, sent them an email with an invitation to check it out and please let me know what they thought. NO response. They probably get deluged. Anyway, I can't afford to pay a marketing company to promote it. Where can I learn how to best do this myself? The content isn't going to help anyone if no one sees it.... Thanks for any leads! Ron
Intermediate & Advanced SEO | | yatesandcojewelers1 -
Sitemap not indexing pages
My website has about 5000 pages submitted in the sitemap but only 900 being indexed. When I checked Google Webmaster Tools about a week ago 4500 pages were being indexed. Any suggestions about what happened or how to fix it? Thanks!
Intermediate & Advanced SEO | | theLotter0 -
What's the best way to phase in a complete site redesign?
Our client is in the planning stages of a site redesign that includes moving platforms. The new site will be rolled out in different phases throughout a period of a year. They are planning to put the new site redesign on a subdomain (i.e. www2.website.com) during the roll out of the different phases while eventually switching the new site back over to the www domain once all the phases are complete. We’re afraid that having the new site on the www2 domain will hurt SEO. For example, if their first phase is rolling out a new system to customize a product design and this new design system is hosted on www2.website.com/customize, when a customer picks a product to customize they’ll be linked to www2.website.com/customize instead of the original www.website.com/customize. The old website will start to get phased out as more and more of the new website is completed and users will be directed to www2. Once the entire redesign is completed, the old platform can be removed and the new website moved back to the www subdomian. Is there a better way of rolling out a website redesign in phases and not have it hosted on a different subdomain?
Intermediate & Advanced SEO | | BlueAcorn0