Protecting sitemaps - Good idea or humbug?
-
Is there a way to protect your sitemap.xml so that only Google can read it and would it make sense to do this?
-
From a hacker's perspective, the first order of business is going to be gathering information on the target. does a hacker or someone with malicious intent gain something in obtaining access to your sitemap?
Yes, they do, and that is more information on the layout of your site. How common would there actually be something on the sitemap that could critically expose you to compromise on your VPS/Shared hosting? Um, probably super ultra rare.
But yes there was one time that I was doing an audit for a company and the sitemap did point to a directory that was vulnerable to directory browsing. Fishing around in the directory, I was able to obtain a picture of a PayPal MasterCard front and back because some idiot snapped pictures of it and uploaded it onto the site.
So there are benefits to hiding it, it's relatively easy to do, but if your lazy and don't want to, chances are your good.
-
Hi Herb,
Thank you for your feedback. I think you are right. We are dealing with very short lived up-to-date information so it is vital that as few sites as possible have the information we have. For this reason I was considering to "hide" our sitemaps. Some of our competitors do that but probably we need to find some other measures to achieve our goal.
Cheers
Thomas -
Hi Thomas;
You have not specified your web server platform, but assuming it is Apache it would be easy to do with a regular expression in your .htaccess
However, I do not see any valid reason for doing so. Your sitemap should be a refection of your public menu and internal public links. So other than making it easier for search and other spiders to crawl your site, it does not expose any information that is not available by other methods. So, best practices say that you should have an accurate site map, and unless you have a reson for hiding it that you did not mention I would not hide it.
I will tell you those that you should not bother putting areas you do not want crawled in your robots.txt file and any of the bad folks will not respect the request.
Take care,
Herb
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Adding your sitemap to robots.txt
Hi everyone, Best practice question: When adding your sitemap to your robots.txt file, do you add the whole sitemap at once or do you add different subcategories (products, posts, categories,..) separately? I'm very curious to hear your thoughts!
Technical SEO | | WeAreDigital_BE0 -
Subpage with own homepage and navigation good or bad?
Hi everybody, I have the following question. At the company I work, we deliver several services. We help people buy the right second hand car (technical inspections). But we also have an import-service. Because those services are so different, I want to split them on our website. So our main website is all about the technical inspections. Then, when you click on import, you go to www.example.com/import. A subpage with it's own homepage en navigation, all about the import service. It's like you have an extra website on the same domain. Does anyone has experience with this in terms of SEO? Thank you for your time! Kind regards, Robert
Technical SEO | | RobertvanHeerde0 -
Are subdomains a good seo strategy for a multistore e-commerce?
Hi there I'm wondering what is the best strategy to work with multi-stores on magento: to use or not to use subdomains? Suppose we have the www.website.com and we configure it to use multistore. The url base will not have the store id on it so it will not be like www.website.com/store1 and www.website.com/store2. It will simply rely on the user session so if we have two categories for each store it will acces using: www.website.com/category1 (for store 1) www.website.com/category2 (for store 2) The homepage will allways be set on www.website.com so we should have a single page for several "home pages" (depending on the user session / store he is accessing). I guess this is not a good option if we want to rank for different keywords (for each store). So I was wondering if it is a good solution to set: store1.website.com store2.website.com This way we have 2 "home pages" each one able to rank. Does it make sense? Is it good or bad for seo? Another option I was considering was: www.website.com (for store 1) store2.website.com (for store 2) store3.website.com (for store 3) www.website.com/blog (for blog) Can this work? Good or bad for seo? best regards
Technical SEO | | qgairsoft0 -
Remove sitemap, effect ranking?
We are considering to remove our sitemap because it doesn't display the right structure. Will it affect current rankings if we remove the sitemap en continuing without a sitemap? Thanks
Technical SEO | | rijwielcashencarry0400 -
#1 on Bing, nowhere on Google. Should be at least top 3\. Any ideas?
I have a page that "should" be top 3 on Google - it's optimised (A on the Moz Pro page grader), it's the most relevant result (it's for an e-book, and the page is the publisher's page for the e-book). Other pages on the site for other books are top of the Google SERPs, and this page itself is top in Bing for the search phrase. The page is https://camphorpress.com/books/formosan-odyssey/ and the keyphrase I want to rank for is "formosan odyssey" (with or without the quotes). Does anyone have any insight as to why it's not ranking in Google? Over-optimised? Duplicate content? Many thanks.
Technical SEO | | C-Tech0 -
Should I include tags in sitemap?
Hello All, I was wondering if you should include tags and categories in your sitemap. In the past on previous blogs I have always left tags and categories out. The reason for this is a good friend of mine who has been doing SEO for a long time and inhouse always told me that this would result in duplicate content. I thought that it would be a great idea to get some input from the SEOmoz community as this obviously has a big affect on your blog and the number of pages indexed. Any help would be great. Thanks, Luke Hutchinson.
Technical SEO | | LukeHutchinson1 -
Reciprocal link building a bad idea?
Reciprical Link Building or one way traffic? Good morning from 8 degrees c but very sunny wetherby UK Regarding link acquisition ive had it in my head that the best idea is to secure inbound links only i.e avoid reciprocal link building so that when your source links to you you dont give them a link back. But is this no longer valid when targeting inbound links to increase rank? Is it ok to link back or will that "leak SEO juice" or whatever we decide to call it this week.
Technical SEO | | Nightwing0 -
Sitemap Creation
Hi I am looking for the best way to generate an XML sitemap for webmaster tools for my website http://www.cheapfindergames.com. I have come across http://www.xml-sitemaps.com/ but it only allows up to 500 links. Is there a PHP script that any experts could share that would create the XML map that I could upload please? Many Thanks
Technical SEO | | ocelot0