External Links from own domain
-
Hi all,
I have a very weird question about external links to our site from our own domain.
According to GWMT we have 603,404,378 links from our own domain to our domain (see screen 1) We noticed when we drilled down that this is from disabled sub-domains like m.jump.co.za.
In the past we used to redirect all traffic from sub-domains to our primary www domain. But it seems that for some time in the past that google had access to crawl some of our sub-domains, but in december 2010 we fixed this so that all sub-domain traffic redirects (301) to our primary domain. Example http://m.jump.co.za/search/ipod/ redirected to http://www.jump.co.za/search/ipod/
The weird part is that the number of external links kept on growing and is now sitting on a massive number.
On 8 April 2011 we took a different approach and we created a landing page for m.jump.co.za and all other requests generated 404 errors. We added all the directories to the robots.txt and we also manually removed all the directories from GWMT.
Now 3 weeks later, and the number of external links just keeps on growing: Here is some stats:
11-Apr-11 - 543 747 534
12-Apr-11 - 554 066 716
13-Apr-11 - 554 066 716
14-Apr-11 - 554 066 716
15-Apr-11 - 521 528 014
16-Apr-11 - 515 098 895
17-Apr-11 - 515 098 895
18-Apr-11 - 515 098 895
19-Apr-11 - 520 404 181
20-Apr-11 - 520 404 181
21-Apr-11 - 520 404 181
26-Apr-11 - 520 404 181
27-Apr-11 - 520 404 181
28-Apr-11 - 603 404 378
I am now thinking of cleaning the robots.txt and re-including all the excluded directories from GWMT and to see if google will be able to get rid of all these links.
What do you think is the best solution to get rid of all these invalid pages.
-
We had 301s for about 6 months, and the old URLs did not disappear from google. Thats why we decided to change them to 404s, with the thinking that Google might remove them quicker. But the number of links from sub-domains just keeps on growing.
I am worried that by having these problem urls listed in the robots.txt actually prevents google from following them and seeing that it should be removed and that it returns a 404
-
Instead of trying to manage a massive 301 list, can you just customize your 404 page to redirect?
{script to test page URL}
$location = "http://www.YourSite.com/";
header("HTTP/1.1 301 Moved Permanently");
header("Location: {$location}");
exit;
}
-
Update:
There are 2 things that still puzzles me with this:
If you go to http://www.google.co.za/search?q=site:jump.co.za+-www&hl=en&rlz=1C1GPCK_enZA426ZA426&prmd=ivns&filter=0&biw=1920&bih=979 you notice all sorts of weird sub-domains, and all of these are invalid and have been removed from GWMT.
If you manage the domain m.jump.co.za on GWMT you also notice that it still reports on keywords, queries and all sorts of data, although the site is disabled and all the URLs generate 404 errors
There is only a few of these weird sub-domains that are causing the problems:
0www.
iiiiiwww.
iwww.
m.
wtfwww.
www.www.
wwww.All these domains feels very fimiliar to me and I am almost 100% sure that its domains that used to test when we found the problem on apache, meaning google took the data from the toolbar queries and probably started indexing these sub-domains. But now I can't get rid of them, and Google seems to be out of control with these.
So the main question is probably, should we just give 404s or should we add to Robots.txt as well?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Linking to my Site so I should Link Back?
I remember hearing a few years ago that it was a good practice to link back to a site that was linking to you. My company's site was referenced and linked to in a news article. The news company has an above average domain authority, which is pretty good for my company's backlink profile. Is it still or was ever a "best practice" to link back to this website/domain? I feel like linking back was a best practice, but when I try to search this, all I get back is backlinking 101 and backlinking articles. Nothing really answering my question straight forward. Thanks for any help.
Technical SEO | | aua0 -
301 - Specific Domain
In Google our main website page url is <cite class="_Rm">www.wheelchairparts.com</cite> it has 25 root domain links & PA 34. When you go to our website from there, the homepage url is http://www.wheelchairparts.com/store/pc/home.asp it only has 3 root domain links & PA 27. Do I need to add a 301 Redirect here? What kind of redirect is on it now? Thanks, - Mike Bean
Technical SEO | | Mike.Bean0 -
Disavow a big part of my external link profile
Hi There, With the latest penguin 3.0 algorithm update (on October 17th,) I noticed a drop in my rankings. Even though I didn’t receive any manual penalty because no messages have been found in WebMaster Tool, I suspect it is an algorithm penalty. For this reason, I definitively decided to clean-up my external link profile. **I am excluding it is a Panda 4.1 penalty because an extensive site structure review has been conducted quite recently. I collected external links from Webmaster Tool and Open Site Explorer. What I found is that 83% of my external links need to be disavowed because the links come either from poor directories or marketing articles that are evidently and specifically written for link building purposes. My questions are: 1) Shall an external link clean-up be set in place anyway although I didn’t receive any penalty message in order to prevent future problems with penguin algorithm? 2) Is it too dangerous to disavow 83% of external links? May such a manoeuvre destroy my actual rankings? Thanks in advance for you advices 🙂
Technical SEO | | Midleton0 -
I have a sub domain that has live content on it but the root domain redirects to another URL. I know this is not great but what are the implications?
I have a subdomain that is populated and has content. The root domain that the sub lives on redirects to an entirely different URL. I am trying to make a case as to why this isn't great besides the fact that it is just weird user experiences. What are the SEO implications etc. Would any equity that gets built up on the subdomain get passed along in the redirect? Or will there be indexation issues with Google? Cheers, Mark
Technical SEO | | mjsikorsky0 -
Domain Name Acquisition
Hi Anyone know how to get contact details for a domain owner beyond those provided by the likes of whois, archive etc etc ? Cheers Dan
Technical SEO | | Dan-Lawrence0 -
Linking to unrelated content
Hi, Just wanted to know, linking to unrelated content will harm the site? I know linking to unrelated content is not good. But wanted to know weather any chances are there or not. I have a site related to health and the other one related to technology. The technology site is too good having PR 6 and very good strong backlinks. And the health related site has very much tough competition, So i wanted to know may be i could link this health site to technology site to get good link from it. Can you suggest me about it. waiting for your replies...
Technical SEO | | Dexter22387874870 -
Old owners links pointing to new owners domain
We have a number of web sites. We recently acquired an excellent domain name, it happened to be owned by one of our competitors. Our competitor has a lot of web sites, each domain having a basic 5 page unoptimized site with one of those pages little more than a link farm. They have over three hundred domains with all but lets say 10 of them consisting of basic 5 page sites with a link "directory" on one of those pages. - the directory page is the same on every single site/domain. One of the links from that directory is going to our newly acquired domain and newly optimized web site. Being new to this, should this pose any kind of concern for us? Thanks in advance!
Technical SEO | | PlasticCards0 -
If you add a no follow to a time sensitive link, will it get picked up as broken link 404 in WMT report?
We have a client who publishes deals that are time sensitive. Links to the deals expire and so Google's crawlers are picking them up and finding a 404 If I no follow them, will the 404's still get picked up and reported in WMT? The same question applies to SEOMoz Pro.
Technical SEO | | Red_Mud_Rookie0