Blocking external links in Robots.txt - need advice on Best Practice
-
I look after an affiliate site that is doing quite well in the search rankings. We've been doing a review of our practices and one thing that has cropped up is our robots.txt.
In it, we disallow Google from crawling external links. This used to be best practice in the affiliate industry a couple of years ago, but I wanted to know if this is still the case, and what the implications are if we were to:
a/ leave it as is?
b/ allow crawling?
Thanks in advance.
-
I'm not sure how you're blocking Google from crawling external links in the robots.txt file--typically you only block them from crawling internal pages on your site.
If you're using a script, though, to track the clicks on external links and that script is running your site (and you're blocking that script in robot.txt), then that still should be fine. You may want to add a "nofollow" tag on those links, though, so you don't end up passing link "credit" or "link juice" to those affiliates (unless you want to do that?).
As far as external links go, though, it's typically okay (and expected) that you link out to other sites.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Can 302 chains (affiliate links) from "toxic" sources hurt you? Or are you "shielded"?
Hi, I'm going through some affiliate links, which send visitors to our website via a chain of several 302 redirects Some of them are relevant links, others perhaps not so much. I know that Google doesn't pass PageRank on 302s... But are they still considered valid links that pass, let's say, "reputation", "relevance", "link neighbourhood" kind of signals? Otherwise put, is a 302 similar to adding the "nofollow" attribute on a link? Sort of? Not at all? More succinctly put, should I be worried about "toxic" sources separated from us by 302 redirect chains? By the way, yes I recognize that (Google's 302 redirect chain handling aside) associating our brand with perhaps what some might consider spammy websites is not in general a good move; I'm concerned with the technical SEO implications here however. In fact, this technical information may very well help drive decisions/policies on where we allow our affiliate advertising to appear. Thanks 🙂 PS - The affiliate company by the way is cj.com if that helps
Affiliate Marketing | | ntcma0 -
Which is the Best Affiliate Marketing Platform in Australia?
We have 1 eCommerce website (www.bannerbuzz.com.au) for printing products like banners, signs, banner stands, teardrop flags and more... We want to start affiliate marketing for our products with CPA model. So, Do anyone have a good experience with Australian affiliate marketing platform. We are having shareasale account for our USA based website, But Shareasale is not that much aggressive in Australia. So we want to finalize 1 platform for affiliate marketing which can have good strength of advertisers and reputation. Thanks,
Affiliate Marketing | | CommercePundit0 -
What is the best way to boost no. of quality sites linking in?
Analysing my site ranking compared to competitors, I see that our weakness is getting others to share our content and link into it via their own blogs and social media - although I would say our social media activity is pretty good. We have also worked on getting listed free on relevant sites related to our product area (learn spanish in spain) in order to increase sites linking but this is still minute compared to our 2 main competitors. (As a note: paid listings are not totally justifiable for us at this point in time but perhaps in the future we will expand on this). Am I missing something here?
Affiliate Marketing | | sparkit0 -
'legitimate' link wheels
I was wondering what SEOMoz' thoughts are on the mega legitimate link wheel sites that are out there. TechMediaNet have been buying up massive news/media sites which arent really monetised (adsense) like http://www.space.com/ http://www.ouramazingplanet.com/ livescience.com and others and generating, admittedly good quality, curated content. Then seeding them with content with backlinks to their money site, toptenreviews.com which in essence is a review site with thousands of pages loaded with affiliate links no better than any of the other site out there. e.g. http://www.livescience.com/9755-bing.html due to the scale of what they're doing pretty much any keyword search i do with review in it (the last thing i wanted to purchase was a usb 3 hub) ends up with toptenreviews.com dominating the serps presumably due to the high PR the viral nature of the media sites are working with. Do you think Google will crack down on this, or do you think it's capitalism in action? TechMediaNet have invested millions in this project and just gotten another $33 million in funding recently.....
Affiliate Marketing | | idimmu0 -
Will giving a blogger an affiliate url to my site reduce her links' seo value to me?
If a blogger were to write a post about one of my products and link to my product page, would giving her an affiliate url to link to rather than the normal link spoil the seo value of the link? So instead of mysite.com/product-one it would be mysite.com/affiliate/blogger/product-one Google would think... this is a transaction not a vote of confidence?
Affiliate Marketing | | Brocberry0 -
Where to look for relevant links
I and many others are targeting keywords such as "free iphone" or "free ps3" as part of incentive marketing schemes that offer rewards for referring new users to businesses. These often seem too good to be true but actually do work (if you don't get caught up in the spam sites). The thing is, what sort of link building should I be looking at once I have my site showing the users how the opportunity works and that it is a legitimate site. I don't want to have free iphone links placed where people may think I'm scamming them, so should I look at press releases, creating infographics or what? I look forward to hearing some of your ideas...
Affiliate Marketing | | GrassRootsSEO0 -
Affiliate links and Panda
Hello all, my question relates to the recent panda changes and the subsequent impact on pages with affiliate links. I have a hub of pages on my website talking about parcel delivery. Each page has unique, quality content and up until about 3 weeks ago they were starting to achieve strong rankings. We then decided to add some affiliate links (just 3) on each of the 5 parcel related pages we have. The rankings for each of these pages have tanked completely; rather than top 20, we're now outside of the top 100 for all of them. When we noticed this we masked the link, changing it to domain.com/index/affiliate; we also no followed the link. Alas this has had minimal effect in restoring the rankings. What i'm now thinking of doing is removing the affiliate links from the parcel landing pages altogether and instead providing users with a no followed link to another page within the site upon which we would have the affiliate links. This page would also be no indexed as well. My question is: Is this a good idea and if so will it restore my previous rankings over time, or have those pages (as a result of the affiliate links) been permanently down graded?? Any assistance would be gratefully received!
Affiliate Marketing | | danielparry0 -
Important question on affiliate links
The website I am working on is http://www.treadmillreviews.com. If you look at any of our affiliate links, they all have an href value of # or something that keeps the user on the page. However, there is an onclick="window.open('affiliate link URL')" code in them that has the affiliate link open in a new window. This website was set up to "hide" the affiliate links in essence this way. I just tried to make a PPC campaign in Google. Got shut down in about 20 minutes for being a "bridge page." Now, all of the content is original and not copied from anywhere. Supposedly Google could not see these links. Yet we got labeled how we did, and now I"m thinking it is causing rankings to hurt a bit. My question is, should I change how all of the links are set up? Should I make the href value be the affiliate link? It is a redirect like CJ.com uses. Should I nofollow those affiliate links? What is the best way to handle this problem? All help is appreciated.
Affiliate Marketing | | DanDeceuster0