Block web archieve/way back machine
-
Hi i want to block web archive/wayback machine from indexing my site and creating a record of it on their database.
Any ideas on how to do this?
Cheers,
Superpak -
You can block Wayback Machine from crawling and creating a record of your site by adding the following to your Robots.txt file:
User-agent: ia_archiver
Disallow: /This will not only stop new records from being created but also stop people viewing what had previously been indexed by Wayback Machine.
More information about this can be found here: https://archive.org/about/exclude.php
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Link backs from same IP? Can it hurt?
We host several client sites on our VPS host. We'd like to add a link back to the footer, however cautious as these are all on the same IP. Will this hurt our SEO?
Intermediate & Advanced SEO | | idlwebinc
Will Google ignore these link backs?0 -
Duplicated content multi language / regional websites
Hi Guys, I know this question has been asked a lot, but I wanted to double check this since I just read a comment of Gianluca Fiorelli (https://moz.com/community/q/can-we-publish-duplicate-content-on-multi-regional-website-blogs) about this topic which made me doubt my research. The case: A Dutch website (.nl) wants a .be version because of conversion reasons. They want to duplicate the Dutch website since they speak Dutch in large parts of both countries. They are willing to implement the following changes: - Href lang tags - Possible a Local Phone number - Possible a Local translation of the menu - Language meta tag (for Bing) Optional they are willing to take the following steps: - Crosslinking every page though a language flag or similar navigation in the header. - Invest in gaining local .be backlinks - Change the server location for both websites so the match there country (Isn't neccessery in my opinion since the ccTLD should make this irrelevant). The content on the website will at least be 95% duplicated. They would like to score with there .be in Belgium and with there .nl in The Netherlands. Are these steps enough to make sure .be gets shown for the quarry’s from Belgium and the .nl for the search quarry’s from the Netherlands? Or would this cause a duplicated content issue resulting in filtering out version? If that’s the case we should use the canonical tag and we can’t rank the .be version of the website. Note: this company is looking for a quick conversion rate win. They won’t invest in rewriting every page and/or blog. The less effort they have to put in this the better (I know it's cursing when talking about SEO). Gaining local backlinks would bring a lot of costs with it for example. I would love to hear from you guys. Best regards, Bob van Biezen
Intermediate & Advanced SEO | | Bob_van_Biezen0 -
Reverting back to old domain name.
I've recently been asked by a client if I can foresee any issues with reverting back to their original domain name. With the original domain name they had a pretty decent DA for their sector which they have now lost. Although I do appreciate that over time this might come back, the CEO is very keen to switch back to the old domain. They do currently have 301 redirects from the old domain to the new and have implemented rel canonical. As yet they have not notified Google of the change of address using Webmaster Tools. Can anyone forsee any issues with returning back to the old domain name? They have only been using the new domain name for a couple of months which currently has a DA for 1.
Intermediate & Advanced SEO | | Macrofireball0 -
What tags/coding are not good for SEO?
what tags/coding are not good for SEO? and also what tags not to include while creating website. For example - I read some where to avoid Span tag.
Intermediate & Advanced SEO | | JordanBrown0 -
RSS and rel = prev/next for pagination
I've noticed on moz report an alert about having to many links on my rss page. http://disneyticketsfree.com/rss/news-updates.html Is using google pagination the way to go? http://googlewebmastercentral.blogspot.com/2011/09/pagination-with-relnext-and-relprev.html If you look at entries they are all about Orlando travel related topics. Thanks to the community in advance.
Intermediate & Advanced SEO | | touristips0 -
Should I Roll Back Domain Change?
A couple years ago I changed domain names and switched platforms for my site. The traffic dropped dramatically (80-90%). I've tried to get inbound links changed, clean up on-page stuff, but nothing is making a big change. I think most of the problem is loss of link juice with the 301 redirects from the old domain to the new one. Would I be risking bigger losses by switching back to the old domain name?
Intermediate & Advanced SEO | | iJeep0 -
Why does this show up in my browser/index2.php
i type is a clients site and BOOM...it suddenly ends with "index2.php" who will link to that? I can't seem to make it not appear...according the template creator... Is this "index2.php" affecting my link juice?
Intermediate & Advanced SEO | | SEObleu.com0 -
Link Juice / Java pop up
Hi all I am a bit unsure of something and would appreciate it if someone could clarify (without the sad trombone hinting that my question is stupid like the last time i asked a question) Our Newsletter was recently posted on a website and i am not sure if the link pointing back is actually passing link juice. When clicking the link, a Java pop up box appears saying "click here to go to authors site" I am wondering if this was implemented to avoid google passing its juice? Or if google can index the pop up and give us credit for the link? Please have a look at the article, and let me know what you guys think? http://www.bestholidaynews.com/adventure-and-activities/africa/our-top-3-overlanding-egypt-trips-2.html Thanks in advance Regards Greg
Intermediate & Advanced SEO | | AndreVanKets0