Changing Links to Spans with Robots.txt Blocked Redirects using Linkify/jQuery
-
Hi,
I was recently penalized most likely because Google started following javascript links to bad neighborhoods that were not no-followed. The first thing I did was remove the Linkify plugin from my site so that all those links would disappear, but now I think I have a solution that works with Linkify without creating crawlable links. I did the following:
-
I blocked access to the Linkify scripts using robots.txt so that Google won't execute the scripts that create the links. This has worked for me in the past with banner ads linking to other sites of mine. At least it appears to work because those sites did not get links from pages running those banners in search console.
-
I created a /redirect/ directory that redirects all offsite URLs. I put a robots.txt block on this directory.
-
I configured the Linkify plugin to parse URLs into span elements instead of a elements and add no follow attributes. They still have an href attribute, but the URLs in the href now point to the redirect directory and the span onclick event redirects the user.
I have implemented this solution on another site of mine and I am hoping this will make it impossible for Google to categorize my pages as liking to any neighborhoods good or bad. Most of the content is UGC, so this should discourage link spam while giving users clickable URLs and still letting people post complaints about people that have profiles on adult websites. Here is a page where the solution has been implemented https://cyberbullyingreport.com/bully/predators-watch-owner-scott-breitenstein-of-dayton-ohio-5463.aspx, the Linkify plugin can be found at https://soapbox.github.io/linkifyjs/, and the custom jQuery is as follows:
jQuery(document).ready(function ($) { 2 $('p').linkify({ tagName: 'span', attributes: { rel: 'nofollow' }, formatHref: function (href) { href = 'https://cyberbullyingreport.com/redirect/?url=' + href; return href; }, events:{ click: function (e) { var href = $(this).attr('href'); window.location.href = href; } } }); 3 });
-
-
I know I was penalized because search console recorded an overnight drop in impressions/clicks of 75%. Never was a problem before. I didn't need to run Screaming Frog or anything to find links that explain the problem. I just browsed some of the user generated content and then once I realized that Google was crawling javascript generated links it all made sense.
Some of the sites people were liking to included: CheaterLand.com, PredatorsWatch.com, DirtyHomeWreckers.com, EscortBabylon.net, SugarDaddyforMe.com, GFEMonkey.com, EscortBabylon.com, CityXGuide.com, and AdultLook.com.
I think as long as Google respects the robots.txt directives I should be just fine. The redirect page itself is blocked by robots.txt so even if Google finds one of those URLs (ex: https://cyberbullyingreport.com/redirect/?url=https://moz.com/community/q/changing-links-to-spans-with-robots-txt-blocked-redirects-using-linkify-jquery) it shouldn't even follow the 302 since the robots.txt file (https://cyberbullyingreport.com/robots.txt) disallows that directory.
-
301 or 302 redirecting well if you’re going to do one you should do 302 but it’s not going to help you much unless you’re going to send the URL to a 410 when something is no followed it is the same thing as what robots.txt will do.
https://support.google.com/webmasters/forum/AAAA2Jdx3sUEbHp0yjgT6c?hl=sv
do you have a report from Google knowing that you have been penalized?
is there anyway you could run Screaming Frog and show some of these URLs that you’re talking about?
respectfully,
Tom
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Is RSS feed syndication an effective link building strategy? Has anyone used it and had success?
This process was recommended to us and I am having trouble understanding exactly how it works. Does this type of link building directly benefit your site or is it an indirect process? Also, can you be penalized for republishing someone's content on your feed?
White Hat / Black Hat SEO | | marketingdepartment.ch0 -
Backlink, how to delete or find who is linking to me?
Hi there guys, Can someone tell me how I go about finding who is linking to my site or how to find backlinks to my site and if it is a spam site or a site I don't know or want linking to me, how to stop them from linking to me and also how to delete their link? Thanks appreciate the time Cheers
White Hat / Black Hat SEO | | edward-may0 -
Link building freelancers or referrals to link building freelancers
Hi, Are there many freelancers in this community that advocates the MOZ linkbuilding philosophies? Or does anyone have references for link building freelancers at a reasonable rate? Thanks, Jack
White Hat / Black Hat SEO | | jackgao840 -
Was I hit by Panda/Payday Loan/Penguin?
Good Mozzing, So, as some of you may know based on my previous post, I am working with an odd situation here. I have taken over an account for a company and the Main domain pretty much falls into the category of everything Google hates. I have suggested to the CEO that the practices they did before me were sorta in the Grayhat realm bordering on Blackhat but I need empirical data before I can make any drastic changes. In May and June of 2013 Panda, Penguin, and Payday Loan all had updates. Our company has nothing do to with porn, apartment rentals, finances, or anything like that, but the SEO methods used were, as I said, questionable. In June of 2013 there was a drop from 8,000 sessions to 5,000 sessions from organic traffic. If I switch over to all referring traffic the loss increases to 11,000 to 7,000 sessions. To me that seems pretty substantial. Not only that, but according to the data we have not been able to recover.There was a steady climb for about 5 months before the drop, and then now we are in this middle ground. I have only been here for about 2 weeks, so the things I have been uncovering are pretty amazing. Is that enough to assume that we were indeed hit by the updates?
White Hat / Black Hat SEO | | HashtagHustler2 -
Removing Unnatural Link Penalties
As soon as I began working in my current position at my current company I noticed my predecessor's tendency towards buying link packages from blackhat companies... I knew we were being penalized, and had to prove to him that we needed to halt those campaigns immediately and try our darndest to remove all poison links from the internet. I did convince him and began the process. There was 57% of our backlinks tied to the same anchor phrase with 836 domains linking to the same phrase, same page. Today there are 643 of those links remaining. So I have hit a large number of them, but not nearly enough. So now I am getting messages from Google announcing that our site has been hit with an unnatural link penalty. I haven't really seen the results of this yet in the keywords I am trying to rank for, but fear it will hurt very soon and know that I could be doing better in the meantime. I really don't know what to do next. I've tried the whole "contact the webmasters" technique and maybe have had 1/100 agree to remove our links. They all want money or don't respond.. Do I really need to use this Disavow tool?
White Hat / Black Hat SEO | | jesse-landry
I hear mixed things about it.. Anybody with experience here like to share their stories? Thanks for the moral support!0 -
Whatever Happened to Text Link Ads?
I've searched the web for any objective articles, good or bad, written about Text Link Ads or Text Link Brokers written in the past two years. Other than the occasional discussion board question, SEOs are silent about these services. I know back in 2006, Rand looked upon them almost favorably. But what has happened since then? Is there any legitimate use for these services anymore (as a link builder)?
White Hat / Black Hat SEO | | 1000Bulbs0 -
How Do You Determine If A Link Is Quality?
What tools and signals do you use to determine if a link is quality or not? How can you tell if a link is going to hurt your ranking?
White Hat / Black Hat SEO | | anchorwave0