Removing/ Redirecting bad URL's from main domain
-
Our users create content for which we host on a seperate URL for a web version. Originally this was hosted on our main domain.
This was causing problems because Google was seeing all these different types of content on our main domain. The page content was all over the place and (we think) may have harmed our main domain reputation.
About a month ago, we added a robots.txt to block those URL's in that particular folder, so that Google doesn't crawl those pages and ignores it in the SERP.
We now went a step further and are now redirecting (301 redirect) all those user created URL's to a totally brand new domain (not affiliated with our brand or main domain).
This should have been done from the beginning, but it wasn't.
Any suggestions on how can we remove all those original URL's and make Google see them as not affiliated with main domain?? or should we just give it the good ol' time recipe for it to fix itself??
-
Yes, that's correct Kurt. We want to disassociate our brand from those pages. Thanks for your FB!
-
Yes, very helpful... Thanks!
-
It sounds to me like you don't want the search engines to know that your moving the content, but rather have them think that you have dropped the pages from your site because you don't want the search engines associating those pages with your site, correct?
If that's the case, then you do want to keep the noindex on the old pages and setup 301 redirects as well. The redirects are for real users who happen to use any links/bookmarks to the old pages. By keeping the old pages noindexed, then hopefully the search engines won't crawl them and won't follow the redirects. I'd also remove the pages from the Google and Bing indexes in their webmaster tools for good measure.
If you are linking from your site to the new location of the user content, you may want to nofollow those links or, better yet, create the links in javascript or something to hide them. If all the links to the content just shift to the new location, Google and Bing may still associate it your site with the new site. Then again, if all the content from the old pages is all the new site, then they may figure it all out anyway.
-
You need to get rid of the robots.txt block on those URLs you want to redirect, Alec.
As it is now with the robots block in place, you've told the search engines NOT to crawl those URLs So it's going to be very difficult for them to discover the 301 redirects and learn that they should be dropping the old URLs form the index. After that, it is just a matter of time. (It can also help to leave those old URLs in the xml sitemap for a while to make it easier for the engines to crawl them and discover the 301s)
If none of those URLs were generating any substantial amount of traffic or incoming links, you could also use Google and Bing Webmaster Tools to request that the pages be removed from the index. This will only really work if the pages are organised in a specific directory, as it would likely take far too long to annotate each URL for removal otherwise.
Hope that helps?
Paul
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why My Website's Rank still in Millions
I am getting enough Traffic on my website on best weed killer on affiliate but Moz still showing its Rank in millions. What would be the best strategy to improve the rankings.???
White Hat / Black Hat SEO | | sarahelen0 -
Moz was unable to crawl your site? Redirect Loop issue
Moz was unable to crawl your site on Jul 25, 2017. I am getting this message for my site: It says "unable to access your homepage due to a redirect loop. https://kuzyklaw.com/ Site is working fine and last crawled on 22nd July. I am not sure why this issue is coming. When I checked the website in Chrome extension it saysThe server has previously indicated this domain should always be accessed via HTTPS (HSTS Protocol). Chrome has cached this internally, and did not connect to any server for this redirect. Chrome reports this redirect as a "307 Internal Redirect" however this probably would have been a "301 Permanent redirect" originally. You can verify this by clearing your browser cache and visiting the original URL again. Not sure if this is actual issue, This is migrated on Https just 5 days ago so may be it will resolved automatically. Not sure, can anybody from Moz team help me with this?
White Hat / Black Hat SEO | | CustomCreatives0 -
How to remove this type of external link from Google
Hello, My website has been hacked few days Before, But after resolved it It is generating bad links, So i am Dis-vowing it , But as it is generating links like this,
White Hat / Black Hat SEO | | innovativekrishna1
http://domain.com/a></p><h1>DIXCEL HS-typeスリットディ
i am Not able to disavow it As it generating Spacing between. So my question is : Is there any Way to remove this Type of link from google???
If any body know Please Let me know, I need Do remove this As soon as possible,
please Help, Thank you0 -
Exact match domain - should i use one
i have the domain "region"familyholidays.co.uk for an upcoming site. i was pleased as its memorable and tells the user what its about. i am targetting keywords such as: region family holidays region family hotels region famliy cottages region family campsites is it something i should avoid because of potential penalties. i will be adding plenty of good content and doing all the offsite things but dont want to start with a handicap with an emd? thanks neil
White Hat / Black Hat SEO | | neilhenderson0 -
G.A. question - removing a specific page's data from total site's results?
I hope I can explain this clearly, hang in there! One of the clients of the law firm I work for does some SEO work for the firm and one thing he has been doing is googling a certain keyword over and over again to trick google's auto fill into using that keyword. When he runs his program he generates around 500 hits to one of our attorney's bio pages. This happens once or twice a week, and since I don't consider them real organic traffic it has been really messing up my GA reports. Is there a way to block that landing page from my overall reports? Or is there a better way to deal with the skewed data? Any help or advice is appreciated, I am still so new to SEO I feel like a lot of my questions are obvious, but please go easy on me!
White Hat / Black Hat SEO | | MyOwnSEO0 -
The differences between XXX.domain.com and domain.com/XXX?
hi guys i would like to know which seo value is better? for example if i would put a link in xxx.domain.com or domain.com/XXX which one will give me a better seo value? does it give the same? assuming that domain.com have a huge PR RANK itself. why do people bother making XXX.domain.com instead? hope for clarification thanks!
White Hat / Black Hat SEO | | andzon0 -
Thousands of 301 redirections - .htaccess alternatives?
Hi guys, I just want to ask if there are other possible issues/problems (other than server load) once we implement 301 redirections for 10,000+ URLs using .htaccess. Are there other alternatives?
White Hat / Black Hat SEO | | esiow20130 -
What to do about all of the other domains we own?
So I had asked this question a while back in a previous thread and thought I had the correct answer to it, but just actually heard differently on a webinar by Dr. Pete. Basically, we have a large number of domains that just replicate our website. Some are brand names, some are exact match keyword domains, some are clever plays on words. This is a tactic that our marketing department thought was a good idea. Obviously its not. My question is - Some of these domains actually have a significant amount of link value coming into them. How people found them I'm not sure, but nonetheless, I want to try to take advantage of the incoming links somehow if possible. Dr. Pete recommended against 301 redirecting back to our main domain all at once because that would be a signal to Google that something fishy is going on. This is what I was going to do, but now I'm really not sure what to do now... If possible, it would be great to get Dr. Pete in this thread to get his comments. I wasn't able to get an answer on the SEO in 2012 Pro Webinar.
White Hat / Black Hat SEO | | CodyWheeler0