Changing Links to Spans with Robots.txt Blocked Redirects using Linkify/jQuery
-
Hi,
I was recently penalized most likely because Google started following javascript links to bad neighborhoods that were not no-followed. The first thing I did was remove the Linkify plugin from my site so that all those links would disappear, but now I think I have a solution that works with Linkify without creating crawlable links. I did the following:
-
I blocked access to the Linkify scripts using robots.txt so that Google won't execute the scripts that create the links. This has worked for me in the past with banner ads linking to other sites of mine. At least it appears to work because those sites did not get links from pages running those banners in search console.
-
I created a /redirect/ directory that redirects all offsite URLs. I put a robots.txt block on this directory.
-
I configured the Linkify plugin to parse URLs into span elements instead of a elements and add no follow attributes. They still have an href attribute, but the URLs in the href now point to the redirect directory and the span onclick event redirects the user.
I have implemented this solution on another site of mine and I am hoping this will make it impossible for Google to categorize my pages as liking to any neighborhoods good or bad. Most of the content is UGC, so this should discourage link spam while giving users clickable URLs and still letting people post complaints about people that have profiles on adult websites. Here is a page where the solution has been implemented https://cyberbullyingreport.com/bully/predators-watch-owner-scott-breitenstein-of-dayton-ohio-5463.aspx, the Linkify plugin can be found at https://soapbox.github.io/linkifyjs/, and the custom jQuery is as follows:
jQuery(document).ready(function ($) { 2 $('p').linkify({ tagName: 'span', attributes: { rel: 'nofollow' }, formatHref: function (href) { href = 'https://cyberbullyingreport.com/redirect/?url=' + href; return href; }, events:{ click: function (e) { var href = $(this).attr('href'); window.location.href = href; } } }); 3 });
-
-
I know I was penalized because search console recorded an overnight drop in impressions/clicks of 75%. Never was a problem before. I didn't need to run Screaming Frog or anything to find links that explain the problem. I just browsed some of the user generated content and then once I realized that Google was crawling javascript generated links it all made sense.
Some of the sites people were liking to included: CheaterLand.com, PredatorsWatch.com, DirtyHomeWreckers.com, EscortBabylon.net, SugarDaddyforMe.com, GFEMonkey.com, EscortBabylon.com, CityXGuide.com, and AdultLook.com.
I think as long as Google respects the robots.txt directives I should be just fine. The redirect page itself is blocked by robots.txt so even if Google finds one of those URLs (ex: https://cyberbullyingreport.com/redirect/?url=https://moz.com/community/q/changing-links-to-spans-with-robots-txt-blocked-redirects-using-linkify-jquery) it shouldn't even follow the 302 since the robots.txt file (https://cyberbullyingreport.com/robots.txt) disallows that directory.
-
301 or 302 redirecting well if you’re going to do one you should do 302 but it’s not going to help you much unless you’re going to send the URL to a 410 when something is no followed it is the same thing as what robots.txt will do.
https://support.google.com/webmasters/forum/AAAA2Jdx3sUEbHp0yjgT6c?hl=sv
do you have a report from Google knowing that you have been penalized?
is there anyway you could run Screaming Frog and show some of these URLs that you’re talking about?
respectfully,
Tom
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Internal Links & Possible Duplicate Content
Hello, I have a website which from February 6 is keep losing positions. I have not received any manual actions in the Search Console. However I have read the following article a few weeks ago and it look a lot with my case: https://www.seroundtable.com/google-cut-down-on-similar-content-pages-25223.html I noticed that google has remove from indexing 44 out of the 182 pages of my website. The pages that have been removed can be considered as similar like the website that is mentioned in the article above. The problem is that there are about 100 pages that are similar to these. It is about pages that describe the cabins of various cruise ships, that contain one picture and one sentence of max 10 words. So, in terms of humans this is not duplicate content but what about the engine, having in mind that sometimes that little sentence can be the same? And let’s say that I remove all these pages and present the cabin details in one page, instead of 15 for example, dynamically and that reduces that size of the website from 180 pages to 50 or so, how will this affect the SEO concerning the internal links issue? Thank you for your help.
White Hat / Black Hat SEO | | Tz_Seo0 -
Magento Temporary Redirects?
Just checked my Crawl insights. I have 1981 on Moz 302 redirects - temp I'm not too familiar with Magento - however site domain has moved from .com to .co and although I have set a 301 redirect on the base domain through hta I am assuming it is also temporary redirecting things in the CMS itself? The temporary directs that the site is creating are still on the new domain - but are really odd! Eg Wishlist, Product compare links .co/wishlist/index/add/product/498/form_key/5e7CQkZ54tMSsJtwAnyone have ideas in regards to this?
White Hat / Black Hat SEO | | Kelly33301 -
HTTPS/SSL and Backlinks
I am planning on moving my site to HTTPS. I brought an expired domain and wondering if moving to HTTPS will affects the previous backlinks? Will I need to do a redirect? Will I lose any link juice? Thanks
White Hat / Black Hat SEO | | wspence150 -
Benefits of having outbound links
Are there any strengths (benefits) in having outbound links within the site regarding SEO? If linking to reputable sites, would that help increase our SEO strength or does that only work if they links back to us?
White Hat / Black Hat SEO | | WebRiverGroup1 -
Is linking out to different websites with the same C-Block IP bad for SEO?
Many SEOs state that getting (too many) links from the same C-Block IP is bad practice and should be avoided. Is this also applicable if one website links out to different websites with the same C-Block IP? Thus, website A, B and C (on the same server) link to website D (different server) could be seen as spam but is this the same when website D links to website A, B and C?
White Hat / Black Hat SEO | | TT_Vakantiehuizen0 -
Linking my pages
Hello everybody, i have a small dilemma and i am not shore what to do. I am (my company) the owner of 10 e-commerce web sites. On every site i have a link too the other 9 sites and i am using an exact keyvoerd (not the shop name).Since the web stores are big and have over a 1000 pages, this means thet all my sites have a lot off inbound links (compared with my competiton). I am woried that linking them all together could be bad from Googles point of wiev. Can this couse a problem for me, should i shange it? Regardes, Marko
White Hat / Black Hat SEO | | Spletnafuzija0 -
When to NOT USE the disavow link tool
Im not here to say this is concrete and should never do this, and please if you disagree with me then lets discuss. One of the biggest things out there today especially after the second wave of Penguin (2.0) is the fear striken web masters who run straight to the disavow tool after they have been hit with Penguin or noticed a drop shortly after. I had a friend who's site who never felt the effects of Penguin 1.0 and thought everything was peachy. Then P2.0 hit and his rankings dropped of the map. I got a call from him that night and he was desperately asking me for help to review his site and guess what might have happened. He then tells me the first thing he did was compile a list of websites back linking to him that might be the issue and create his disavow list and submitted it. I asked him "How long did you research these sites before you came the conclusion they were the problem?" He Said "About an hour" Then I asked him "Did you receive a message in your Google Webmaster Tools about unnatural linking?" He Said "No" I said "Then why are you disavowing anything?" He Said "Um.......I don't understand what you are saying?" In reading articles, forums and even here in the Moz Q/A I tend to think there is some misconceptions about the disavow tool from Google that do not seem to be clearly explained. Some of my findings with the tool and when to use it is purely based on logic IMO. Let me explain When to NOT use the tool If you spent an hour reviewing your back link profile and you are to eager to wait any longer to upload your list. Unless you have less than 20 root domains linking to you, you should spend a lot more than an hour reviewing your back link profile You DID NOT receive a message from GWT informing you that you had some "unnatural" links Ill explain later If you spend a very short amount of time reviewing your back link profile. Did not look at each individual site linking to you and every link that exists, then you might be using it WAY TO SOON. The last thing you want to do is disavow a link that actually might be helping you. Take the time to really look at each link and ask your self this question (Straight from the Google Guidelines) "A good rule of thumb is whether you'd feel comfortable explaining what you've done to a website that competes with you, or to a Google employee" Studying your back link profile We all know when we have cheated. Im sure 99.9% of all of us can admit to it at one point. Most of the time I can find back links from sites and look right at the owner and ask him or her "You placed this back link didn't you?" I can see the guilt immediately in their eyes 🙂 Remember not ALL back links you generate are bad or wrong because you own the site. You need to ask yourself "Was this link necessary and does it apply to the topic at hand?", "Was it relevant?" and most important "Is this going to help other users?". These are some questions you can ask yourself before each link you place. You DID NOT receive a message about unnatural linking This is were I think the most confusing takes place (and please explain to me if I am wrong on this). If you did not receive a message in GWT about unnatural linking, then we can safely say that Google does not think you contain any "fishy" spammy links in which they have determined to be of a spammy nature. So if you did not receive any message yet your rankings dropped, then what could it be? Well it's still your back links that most likely did it, but its more likely the "value" of previous links that hold less or no value at all anymore. So obviously when this value drops, so does your rank. So what do I do? Build more quality links....and watch you rankings come back 🙂
White Hat / Black Hat SEO | | cbielich1 -
Whatever Happened to Text Link Ads?
I've searched the web for any objective articles, good or bad, written about Text Link Ads or Text Link Brokers written in the past two years. Other than the occasional discussion board question, SEOs are silent about these services. I know back in 2006, Rand looked upon them almost favorably. But what has happened since then? Is there any legitimate use for these services anymore (as a link builder)?
White Hat / Black Hat SEO | | 1000Bulbs0