Find archived sitemap of a website that no longer exists
-
I am trying to figure out the site structure of a website and the urls of all the pages. Normally this would be easy but a couple of months ago the website went down and I don't think it will ever come back. Any help would be appreciated.
-
Use the internet archive (wayback machine) which effectdigital mentioned above, to find the /robots.txt file from the desired date. In that file you should find the referenced sitemap file (assuming the site properly included its sitemap reference in its robot.txt file). Then you can use the same process to request the sitemap file which was referenced in the robots.txt file.
-
Hi Effect,
Does your second link automatically provider the sitemaps available, or does the user still need to "know" or be able to guess where they might be e.g /sitemap.xml?
Nick
-
You can use this site to see legacy site-maps for some websites (though they may be partial or incomplete):
For example, check these sitemap results:
For smaller sites, the results are much easier to look at.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What tool can I use to find the top traffic-driving keyword for a batch of pages from multiple sites?
I thought I might be able to use Ahref's Batch Analysis for this, but that just gives stats on backlinks. I have a several lists of thousands of pages scraped for particular sets of keywords, but what I need is some way to automate fetching the biggest keyword that each page ranks for - biggest being the keyword that is estimated to drive the most organic traffic. Is there a tool out there that does something like this?
Competitive Research | | helenlorettahasan0 -
Best way to find the highest ranking pages for a message board?
What is the best tool or method to find the highest ranking posts and content on a message board or forum? What is the best tool or method for finding the highest ranking keywords for a message board or forum?
Competitive Research | | Hondaspeder0 -
Why is my website not in the top 3 if our Moz statistics are better?
Hi, We've been using Moz, for our E-commerce sites, for some time now and are improving all of our statistics, be it on-page for specific keywords or general crawl errors, but I've not been seeing a lot of change in our rankings... (we've moved from place 7 to 6 on our main keyword). I'm thinking of looking at our backlinks, through Link Detox, after we finish our Moz optimisation to see if anything weird is going on with that. Are there any other areas I should be looking at? any other clues as to why we're not ranking higher than our competitors. Even if our Moz statistics are better and our Market Samurai values are similar? Thank you very much! Alexander
Competitive Research | | WebmasterAlex1 -
Thoughts on Nofollow My Account / Shopping Cart Pages on an eCommerce Website
I recently noticed most of my competitors (eCommerce sites) have linked to pages within their sites that do not always contain product information (my account, shopping cart, etc) using rel="nofollow". My site does not currently do this. Are there any advantages to using rel="nofollow" on similar pages on our site? Any disadvantages?
Competitive Research | | Gordian0 -
What is a recommended sitemap generator to use for both us and our clients?
Right now we are using xml-sitemaps.com however it looks like it isn't generating an accurate or lengthy sitemap. Do you have any recommendations?
Competitive Research | | MonsterWeb280 -
Websites like these..
Hi, I'm new to link building. I have done some research of my competitors and have found that many of them are being linked to sites like this http://www.worldofwondersvu.com/ As you can see, this website looks like a business website, but if you view the content, it's spammed with external links. I'm just wondering what method they are going about to get these links, because most of the websites that are linking to them are simular to the one I posted above! Thanks 🙂
Competitive Research | | LTTVending740 -
Query for checking is a link to domain A already exists on domain B
Hi, I was wondering if anyone can help. I need to have a simple check where I have 2 domains, and I can check of there are any links from domain A to domain B. Does anyone what would be the best query for this and if you would use google, bing, yahoo or other SE. Many thanks
Competitive Research | | James770 -
Website analysis automation
I need a tool that will input a list of URLs and produce a spreadsheet with link metrics for all sites including PageRank, Domain Authority, Page Authority, Social data, etc..i know that OSE will do this for 5 sites at a time but I need to compare more. At lest 50 at a time... I bought Buzzstream thinking this would work, but it doesn't. does anybody have any good advice? Thanks!!
Competitive Research | | znotes0