External Links from own domain
-
Hi all,
I have a very weird question about external links to our site from our own domain.
According to GWMT we have 603,404,378 links from our own domain to our domain (see screen 1) We noticed when we drilled down that this is from disabled sub-domains like m.jump.co.za.
In the past we used to redirect all traffic from sub-domains to our primary www domain. But it seems that for some time in the past that google had access to crawl some of our sub-domains, but in december 2010 we fixed this so that all sub-domain traffic redirects (301) to our primary domain. Example http://m.jump.co.za/search/ipod/ redirected to http://www.jump.co.za/search/ipod/
The weird part is that the number of external links kept on growing and is now sitting on a massive number.
On 8 April 2011 we took a different approach and we created a landing page for m.jump.co.za and all other requests generated 404 errors. We added all the directories to the robots.txt and we also manually removed all the directories from GWMT.
Now 3 weeks later, and the number of external links just keeps on growing: Here is some stats:
11-Apr-11 - 543 747 534
12-Apr-11 - 554 066 716
13-Apr-11 - 554 066 716
14-Apr-11 - 554 066 716
15-Apr-11 - 521 528 014
16-Apr-11 - 515 098 895
17-Apr-11 - 515 098 895
18-Apr-11 - 515 098 895
19-Apr-11 - 520 404 181
20-Apr-11 - 520 404 181
21-Apr-11 - 520 404 181
26-Apr-11 - 520 404 181
27-Apr-11 - 520 404 181
28-Apr-11 - 603 404 378
I am now thinking of cleaning the robots.txt and re-including all the excluded directories from GWMT and to see if google will be able to get rid of all these links.
What do you think is the best solution to get rid of all these invalid pages.
-
We had 301s for about 6 months, and the old URLs did not disappear from google. Thats why we decided to change them to 404s, with the thinking that Google might remove them quicker. But the number of links from sub-domains just keeps on growing.
I am worried that by having these problem urls listed in the robots.txt actually prevents google from following them and seeing that it should be removed and that it returns a 404
-
Instead of trying to manage a massive 301 list, can you just customize your 404 page to redirect?
{script to test page URL}
$location = "http://www.YourSite.com/";
header("HTTP/1.1 301 Moved Permanently");
header("Location: {$location}");
exit;
}
-
Update:
There are 2 things that still puzzles me with this:
If you go to http://www.google.co.za/search?q=site:jump.co.za+-www&hl=en&rlz=1C1GPCK_enZA426ZA426&prmd=ivns&filter=0&biw=1920&bih=979 you notice all sorts of weird sub-domains, and all of these are invalid and have been removed from GWMT.
If you manage the domain m.jump.co.za on GWMT you also notice that it still reports on keywords, queries and all sorts of data, although the site is disabled and all the URLs generate 404 errors
There is only a few of these weird sub-domains that are causing the problems:
0www.
iiiiiwww.
iwww.
m.
wtfwww.
www.www.
wwww.All these domains feels very fimiliar to me and I am almost 100% sure that its domains that used to test when we found the problem on apache, meaning google took the data from the toolbar queries and probably started indexing these sub-domains. But now I can't get rid of them, and Google seems to be out of control with these.
So the main question is probably, should we just give 404s or should we add to Robots.txt as well?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Blog.domain or domain.com/blog
My client can't do domain.com/blog because he's on wix. I'm thinking blog.domain.com. Do you have any resources for the pros and cons of this? I understand that google looks at them very similarly now, is that true for google +?
Technical SEO | | tylerfraser0 -
How much domain authority is passed on through a link from a page with low authority?
Hello, Let's say that there is a link to site A from site B. The domain authority of site B is 85, but the link is on a page that has a page authority of only 1. Does much authority get passed along from site B to site A? (Let's assume site A has a domain authority of 35, if that's relevant.) Thank you!
Technical SEO | | nyc-seo0 -
Link profile
Hi All, I am doing a link profile audit I have few questions 1. Should i stop worrying about backlinks that i once had and now the websites is down or page is 404 2. The link is nofollow Also i have 60% of my site links few root link and many articles/blogs links pasted in sites without any anchor text, should i worry about them? Thanks
Technical SEO | | mtthompsons0 -
We just recently moved site domains, and I tried to set up a new campaign for the new root domain, but it threw an error?
It threw an error saying we cannot access the SERPs of this site? Any reason why? It is an https:// site instead of the http://, but even our older domain had an https://
Technical SEO | | josh1230 -
Bad link profile?
Hi Mozzers! We have recently been handed this client due to the former SEO company building up a bad link profile, which resulted in the site dropping off the search results all together. Forcing them to get a new domain. This happened in July last year and we are unsure whether it would be wise to submit a reconsideration request and then 301 their old sites pages to the new domain. Basically I'm asking whether you can spot any spammy links being built in their profile. Here is the old domain: http://www.claimssolicitors.co.uk/ It would be great if you could help me out! 🙂 Thanks
Technical SEO | | Webrevolve0 -
Redirect non www. domain to WWW. domain for established website?
Hey guys, The website in question has been online for more than 5 years but there are still 2 versions of the website. Both versions are indexed by Google and of course, this will result in duplicate content. Is it necessary to redirect the non-www domain to the www. domain. What are the cons and advantages? Will the www. links replace the non-www links when it comes to keyword rankings? Thanks.
Technical SEO | | BruLee0 -
Domain redirect
Recently we launched a site under a new domain, the site is doing well under the URL. Client calls me today and would like to have another domain he owns point to the new site. The domain he has has no history and no content. He is under the impression that people are looking for him by typing in www.domainxyz.com. I attempted to explain otherwise to him, but I lost. Question, what are the drawbacks of taking this domin and doing a perm redirect via . Httpaccess file?
Technical SEO | | VanadiumInteractive0 -
Onpage linking
On my homepage, I currently link to about 40 internal pages. I'm considering altering the internal linking structure to have 50-100 links on the 2nd level pages. If I was to do this, I'd only need 8 homepage links. Do you think the 8 pages linked from the homepage would go up in the SERPs as the pagerank would be less diluted? I've heard so many mixed views on this. Be interested to see what people here think. Thanks, Pete
Technical SEO | | PeterM220