Suggestions on Link Auditing a 70,000 URL list?
-
I have a website with nearly 70,000 incoming links, since its a somewhat large site that has been online for 19 years.
The rate I was quoted for a link audit from a reputable SEO professional was $2 per, and clearly I don't have $140,000 to spend on a link audit
!!
I was thinking of asking you guys for a tutorial that is the Gold Standard for link auditing checklists - and do it myself. But then I thought maybe its easier to shorten the list by knocking out all the "obviously good" links first. My only concern is that I be 100% certain they are good links.
Is there an "easiest approach" to take for shortening this list, so I can give it to a professional to handle the rest?
-
Hi! - I wrote this guide a few years ago on penalty recovery which may help you as it contains a lot of methods around auditing the links - https://moz.com/blog/ultimate-guide-to-google-penalty-removal
If we were to approach a product with 70k URLs. We'd do the following steps:
- Pull all the URLs into a Spreadsheet
- Split the URLs into domains
- Filter the URLs are search for common spammy words. e.g 'Link', 'Best', 'Free', 'Cheap', 'Dir', 'SEO' etc (mark as spam accordingly)
- Run contact finding across all URLs using a tool such as URL Profiler with Whois Lookups
- Filter by contact name and find duplicates (mark as spam accordingly)
- Filter by website type and mark as spam accordingly
- Manually check remaining links
By working through by domain, you'll rule out thousands of spammy links very quickly. Though 70k will ultimately take a few solid days of work.
Hope this helps,
Lewis
-
Have you looked at www.monitorbacklinks.com, good tool.
-
Hello,
Although it's important to do a link audit if you feel you have been penalized, for some sites a link audit isn't necessary. With that being said, and you feel you need a link audit there are a few options. Ideally, you would go through each link and review it to see how it may be impacting your site, but often site owners don't have the time to do this.
- Review obvious links - Grab 50-100 links at a time and do a quick glance at each one to determine if it should be on a list of potentially bad links. This way you can quickly overlook links you know are not hurting your rankings. Over time you can slowly tackle your list and hammer out which links are bad.
- Focus on spam analysis links - Run your site through Moz open site explorer and review the spam analysis. Now you're not going to get every single link here, but you can get an idea on what links are lower quality.
- Look into other companies - $2 per link is quite high, and there are other companies out there that will do a link audit, removal, and disavow for much less. If you would like a quote please contact us. Look into multiple options, don't get sold on just what one place tells you.
Hope this is helpful, if you have any additional questions please feel free to ask.
Chris
-
$2 per link is very expensive when you are looking at so many, especially as there is a big part of this that can be automated (hint: This should cost you no more than about $5-$10k if outsourced).
Linda has given you some good tips there, but I do agree that you need to tread carefully because you can often go too far and end up jumping out of the frying pan and into the fire.
It really does help to first gather all of the links from as many sources as you can and as already mentioned, create your de-dupe list. Depending on who you speak to at this point, there are different ways to go through the data and start to segment the links into those you know that are dangerous, those that are perhaps a bit of a grey area, and those that are safe.
Cheers,
Andy
-
I concentrate on the "most normal or typical sites will not need to use this tool" part, myself. (Though it sounds like you may not fall into that category.)
So then it's back to downloading as comprehensive a list of links as you can by using various sources and looking them over. (Also, in the past I have used LinkResearchTools to get an overview--it isn't cheap but it is a lot less than $140,000.)
-
Yes. We have confirmed with Sucuri that there was a concerted, intentional spam campaign against our site in 2013 that has since destroyed our rankings. Though Google hasn't given us any warnings, Sucuri had us on a blacklist because of it, and was kind enough to remove us without any cost or obligation on our part to sign up. They also provided us with a list of some of the most offending links so I could disavow them.
With up to 70,000 total, I am confident there are more, and to be honest, I see no reason to "leave some". Or leave any. I believe Google's warning should focus on this part: "...if used incorrectly". That means ... simply use it correctly. And disavow bad links, period. That's my take at least.
-
First, are you sure you need a link audit? Google is pretty good at ignoring regular spammy links that get picked up over time by large sites, as they say in their "Disavow backlinks" help page.
If you think there is a cause for concern, Moz's own Open Site Explorer can give you a list of incoming links that includes a spam score for those links, which can be used as a first pass.
The general drill for a manual link audit is to find all of the links you can (search console, moz, ahrefs, majestic, etc.) and create a de-duped list. From there, the "definitely good links" are usually easy to spot--you will recognize them from your industry or from other authoritative sources. And you will probably recognize the spammy "Get Rich/Viagra" backlinks as well. (If you sort your list by domain, it is easier to pick them out as a group.)
The rest are the ones to look at more closely.
But as I said to start, unless you think you are being penalized, tread lightly when it comes to disavowals.
To quote from Google [about disavowal]:
"This is an advanced feature and should only be used with caution. If used incorrectly, this feature can potentially harm your site’s performance in Google’s search results. We recommend that you disavow backlinks only if you believe you have a considerable number of spammy, artificial, or low-quality links pointing to your site, and if you are confident that the links are causing issues for you. In most cases, Google can assess which links to trust without additional guidance, so most normal or typical sites will not need to use this tool."
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How Many Links to Disavow at Once When Link Profile is Very Spammy?
We are using link detox (Link Research Tools) to evaluate our domain for bad links. We ran a Domain-wide Link Detox Risk report. The reports showed a "High Domain DETOX RISK" with the following results: -42% (292) of backlinks with a high or above average detox risk
Intermediate & Advanced SEO | | Kingalan1
-8% (52) of backlinks with an average of below above average detox risk
-12% (81) of backlinks with a low or very low detox risk
-38% (264) of backlinks were reported as disavowed. This look like a pretty bad link profile. Additionally, more than 500 of the 689 backlinks are "404 Not Found", "403 Forbidden", "410 Gone", "503 Service Unavailable". Is it safe to disavow these? Could Google be penalizing us for them> I would like to disavow the bad links, however my concern is that there are so few good links that removing bad links will kill link juice and really damage our ranking and traffic. The site still ranks for terms that are not very competitive. We receive about 230 organic visits a week. Assuming we need to disavow about 292 links, would it be safer to disavow 25 per month while we are building new links so we do not radically shift the link profile all at once? Also, many of the bad links are 404 errors or page not found errors. Would it be OK to run a disavow of these all at once? Any risk to that? Would we be better just to build links and leave the bad links ups? Alternatively, would disavowing the bad links potentially help our traffic? It just seems risky because the overwhelming majority of links are bad.0 -
Huge problem : All our Internal Links Dropped from 9.000 to 0.What happened?
Hi, I just noticed a huge large problem in our rankings. Our rankings suddenly dropped for more than 50 %. Of course, I immediately started to research the issue. And under Links, I found that we somehow lost all of our internal links! They have dropped from 9k to 0. Now, I am sure that we do have some internal links on our site ( since I put them there myself). Could you please tell me what is going on and how I can fix this issue? Our site is 1solarsolution.com and I will also attach screenshots bellow from Link Explorer, thank you. Fr08UGe
Intermediate & Advanced SEO | | alisamana0 -
How to determine the value of these links?
Hi Guys, How can you determine the value of external links which are deep inside a website. Two examples: http://www.sheknows.com/community/home/ten-tips-buy-car-insurance Two sub-folders deep. http://www.dogfoodhowto.com/899/whats-the-best-puppy-food-for-cockapoo-puppy-at-home.html One sub-folder deep. These links are clearly far from the homepage, so was wondering if they are worthless or how can you determine the value of them? Cheers.
Intermediate & Advanced SEO | | nattyhall0 -
Contextual links (is this screen shot considered contextual /editorial links ?)
Hello, Is the screen shot below considered contextual ?https://imgur.com/a/mrbQq and does it have any value or no value What is the value on a scale from 0 to 10 (if you know) of a contextual link versus non contextual links. Thank you, mrbQq
Intermediate & Advanced SEO | | seoanalytics0 -
Problem with internal links
Hello! Our domain, http://www.unionroom.com/, is having a strange issue with OSE in that it is telling us our internal pages aren't linking to one another. An example of this is that it is showing our About page ( http://www.unionroom.com/about/ ) only having three links, but this link appears twice on every single page on the website (~200 pages) in the header and footer. We've hung around for a little while to see if OSE would correct itself, but it hasn't and this now suggests that it may be an issue with our in-linking structure. Can anyone spot any issues with our build? The rest of the websites that we produce, that are all built in the same way, all have healthy internal linking structures according to OSE. Very confusing! Any help would be greatly appreciated.
Intermediate & Advanced SEO | | unionroom0 -
Technical Question on Image Links - Part of Addressing High Number of Outbound Links
Hi - I've read through the forum, and have been reading online for hours, and can't quite find an answer to what I'm searching for. Hopefully someone can chime in with some information. 🙂 For some background - I am looking closely at four websites, trying to bring them up to speed with current guidelines, and recoup some lost traffic and revenue. One of the things we are zeroing in on is the high amount of outbound links in general, as well as inter-site linking, and a nearly total lack of rel=nofollow on any links. Our current CMS doesn't allow an editor to add them, and it will require programming changes to modify any past links, which means I'm trying to ask for the right things, once, in order to streamline the process. One thing that is nagging at me is that the way we link to our images could be getting misconstrued by a more sensitive Penguin algorithm. Our article images are all hosted on one separate domain. This was done for website performance reasons. My concern is that we don't just embed the image via , which would make this concern moot. We also have an href tag on each to a 'larger view' of the image that precedes the img src in the code, for example - We are still running the numbers, but as some articles have several images, and we currently have about 85,000 articles on those four sites... well, that's a lot of href links to another domain. I'm suggesting that one of the steps we take is to rel=nofollow the image hrefs. Our image traffic from Google search, or any image search for that matter, is negligible. On one site it represented just .008% of our visits in July. I'm getting a little pushback on that idea as having a separate image server is standard for many websites, so I thought I'd seek additional information and opinions. Thanks!
Intermediate & Advanced SEO | | MediaCF0 -
Linking across categories
On a website when I link across in the same category should all the categories all pear on each page. Let's say I have 6 categories and 6 pages should I have the 6 links on all the pages ( such as A, B, C, D, E, on page 1 ( let's imagine this page is page F ), then on page A have link B, C D, E, F and so on for the 6 pages ( meaning all the links appear on all the pages across the category ) or should i just have let's say 3 links on page 1 ( link A, B, C ) , then link ( D, E, F ) on page 2, then A, E, F on page 3, link B, C F on page 4 and so on... ( which means that i vary the links that appear and that it is naturally ( at least I think ) going to boost the link that appears the most of the 6 pages ? I hope this is not too confusing, Thank you,
Intermediate & Advanced SEO | | seoanalytics0 -
Lots of incorrect urls indexed - Googlebot found an extremely high number of URLs on your site
Hi, Any assistance would be greatly appreciated. Basically, our rankings and traffic etc have been dropping massively recently google sent us a message stating " Googlebot found an extremely high number of URLs on your site". This first highligted us to the problem that for some reason our eCommerce site has recently generated loads (potentially thousands) of rubbish urls hencing giving us duplication everywhere which google is obviously penalizing us with in the terms of rankings dropping etc etc. Our developer is trying to find the route cause of this but my concern is, How do we get rid of all these bogus urls ?. If we use GWT to remove urls it's going to take years. We have just amended our Robot txt file to exclude them going forward but they have already been indexed so I need to know do we put a redirect 301 on them and also a HTTP Code 404 to tell google they don't exist ? Do we also put a No Index on the pages or what . what is the best solution .? A couple of example of our problems are here : In Google type - site:bestathire.co.uk inurl:"br" You will see 107 results. This is one of many lot we need to get rid of. Also - site:bestathire.co.uk intitle:"All items from this hire company" Shows 25,300 indexed pages we need to get rid of Another thing to help tidy this mess up going forward is to improve on our pagination work. Our Site uses Rel=Next and Rel=Prev but no concanical. As a belt and braces approach, should we also put concanical tags on our category pages whereby there are more than 1 page. I was thinking of doing it on the Page 1 of our most important pages or the View all or both ?. Whats' the general consenus ? Any advice on both points greatly appreciated? thanks Sarah.
Intermediate & Advanced SEO | | SarahCollins0