Removing/ Redirecting bad URL's from main domain
-
Our users create content for which we host on a seperate URL for a web version. Originally this was hosted on our main domain.
This was causing problems because Google was seeing all these different types of content on our main domain. The page content was all over the place and (we think) may have harmed our main domain reputation.
About a month ago, we added a robots.txt to block those URL's in that particular folder, so that Google doesn't crawl those pages and ignores it in the SERP.
We now went a step further and are now redirecting (301 redirect) all those user created URL's to a totally brand new domain (not affiliated with our brand or main domain).
This should have been done from the beginning, but it wasn't.
Any suggestions on how can we remove all those original URL's and make Google see them as not affiliated with main domain?? or should we just give it the good ol' time recipe for it to fix itself??
-
Yes, that's correct Kurt. We want to disassociate our brand from those pages. Thanks for your FB!
-
Yes, very helpful... Thanks!
-
It sounds to me like you don't want the search engines to know that your moving the content, but rather have them think that you have dropped the pages from your site because you don't want the search engines associating those pages with your site, correct?
If that's the case, then you do want to keep the noindex on the old pages and setup 301 redirects as well. The redirects are for real users who happen to use any links/bookmarks to the old pages. By keeping the old pages noindexed, then hopefully the search engines won't crawl them and won't follow the redirects. I'd also remove the pages from the Google and Bing indexes in their webmaster tools for good measure.
If you are linking from your site to the new location of the user content, you may want to nofollow those links or, better yet, create the links in javascript or something to hide them. If all the links to the content just shift to the new location, Google and Bing may still associate it your site with the new site. Then again, if all the content from the old pages is all the new site, then they may figure it all out anyway.
-
You need to get rid of the robots.txt block on those URLs you want to redirect, Alec.
As it is now with the robots block in place, you've told the search engines NOT to crawl those URLs So it's going to be very difficult for them to discover the 301 redirects and learn that they should be dropping the old URLs form the index. After that, it is just a matter of time. (It can also help to leave those old URLs in the xml sitemap for a while to make it easier for the engines to crawl them and discover the 301s)
If none of those URLs were generating any substantial amount of traffic or incoming links, you could also use Google and Bing Webmaster Tools to request that the pages be removed from the index. This will only really work if the pages are organised in a specific directory, as it would likely take far too long to annotate each URL for removal otherwise.
Hope that helps?
Paul
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why isn't a 301 redirect removing old style URLs from Google's index?
I have two questions:1 - We changed the URL structure of our site. Old URLs were in the format of kiwiforsale.com/used_fruit/yummy_kiwi. These URLs are 301 redirected to kiwiforsale.com/used-fruit/yummy-kiwi. We are getting duplicate content errors in Google Webmaster Tools. Why isn't the 301 redirect removing the old style URL out of Google's index?2 - I tried to remove the old style URL at https://www.google.com/webmasters/tools/removals, however I got the message that "We think the image or web page you're trying to remove hasn't been removed by the site owner. Before Google can remove it from our search results, the site owner needs to take down or update the content."Why are we getting this message? Doesn't the 301 redirect alert Google that the old style URL is toast and it's gone?
White Hat / Black Hat SEO | | CFSSEO0 -
Link Building / Link Removal
Hey, I'm in the process of learning SEO, or attempting to, for my company and am chipping away at the process ever so slowly! How can I tell if a site that links to my company's site, www.1099pro.com, has a negative effect on my page/domain authority? Also, if a page doesn't show up in the search rankings at all for it's keywords when it really should (i.e it has the exact keywords and page/domain authority far surpasses even the top results) how can I tell if Google has removed the page from its listing and why? Thanks SEO Gurus
White Hat / Black Hat SEO | | Stew2220 -
Domain change after over optimisation
Hi Guys, Wanted to get some second opinions. I have a client who worked with someone on their SEO for about 15months, during which time they reaped the benefits of being top of Googles organic rankings for their main keyword. Unfortunately, the link building that was completed was terrible with no care taken for future updates, over optimised anchor text links on totally irrelevant websites. Contacting webmasters in order for links to be taken down was an impossible task, these websites were pure spam sites which I doubt anyone oversees with any care. So went the disavow route, it caused a slight increase, but the way Google indexes the website and its pages is still all over the place. We have a good new content marketing plan in place for the next 12 months, and already have some great in depth articles, videos and interactive infographics created, but the problem is my client really needs some quick wins and cannot risk the domain never recovering fully from the issues. They own the .com version of the .co.uk they have so my thoughts at this point are to move them to the .com, and the new content strategy can be worked on as if its a new project. For the keywords they target there are sites on the 1st and 2nd page with DA of 16-18, so it wont take long with the new content we have to get them back to a level playing field with the competition. The one worry I do have is Google seeing the .co.uk go down, and the .com appear with the same website on and looking down on the "start from scratch" method.... does anyone know of any examples when they have acted on this? We started an AdWords account (very small spend) and this has been going to the .co.uk, if we start a new one, with same billing info going to the .com could that also cause problems? So basically, because of my clients situation, and the level the competition is it, moving to the .com seems like the best move, its just whether there is any risk involved. I've never read of there being any, but thought I'd get some opinions before comitting to it. By the way, I'm only seeing this as a feasible option as they can still keep branded traffic due to buying both domains at the same time. Thanks in advance.
White Hat / Black Hat SEO | | gamnaking10 -
Are multiple links on the same domain worth pursuing?
Let's say you get a guest post on a blog that links to your site for $100. How much is the link from another guest post on the same domain worth? $90? $50? Does each additional link from the same domain lose its SEO impact? What if one blogger loves your content and is willing to post 10+ of your blog posts with links to your site in each - is that worth pursuing just from an SEO standpoint (I know it can be a good branding opportunity if the readership is right)?
White Hat / Black Hat SEO | | pbhatt0 -
LOCAL SEO / Ranking for the difficult 'service areas' outside of the primary location?
It's generally not too hard to rank in Google Places and organically for your primary location. However if you are a service area business looking to rank for neighboring cities or service areas, Google makes this much tougher. Andrew Shotland mentions the obvious and not so obvious options: Service Area pages ranking organically, getting a real/virtual address, boost geo signals, and using zip codes instead of service area circle. But I am wondering if anyone had success with other methods? Maybe you have used geo-tagging in a creative way? This is a hurdle that many local business are struggling with and any experience or thoughts will be much appreciated
White Hat / Black Hat SEO | | vmialik1 -
Can I Point Multiple Exact Match Domains to a Primary Domain? (Avoiding Duplicate Content)
For example, lets say I have these 3 domains: product1.com product2.com product.com The first 2 domains will have very similar text content, with different products. The product.com domain will be similar content, with all of the products in one place. Transactions would be handled through the Primary domain (product.com) The purpose of this would be to capitalize on the Exact match domain opportunities. I found this seemingly old article: http://www.thesitewizard.com/domain/point-multiple-domains-one-website.shtml The article states that you can avoid duplicate content issues, and have all links attributed to the Primary domain. What do you guys think about this? Is it possible? Is there a better way of approaching this while still taking advantage of the EMD?
White Hat / Black Hat SEO | | ClearVisionDesign0 -
Is there any value in "starting from scratch" on a new domain?
Hi, Our ecommerce store - we have had some duplicate content issues and they have been corrected, but of course, Google takes time to pick up on these. Our link profile is very poor, so we wont lose a lot by going to a new domain in that sense. My question is, in what instances is it worthwhile starting under a new domain? And in which not? Presumably you can also 301 the whole site - when is it worth doing this or not? Thanks, Ben
White Hat / Black Hat SEO | | bjs20100 -
Negative SEO on my website with paid +1's
Hi guys, I need a piece of advice. Some scumbag played me quite well with paid +1's on my two articles and now I'm in a problem.
White Hat / Black Hat SEO | | Fastbridge
http://sr.stateofseo.com/seo-vesti/google-implementacija-ssl-protokola-not-provided-problem/
http://sr.stateofseo.com/napredni-seo/najnovije-promene-google-panda-algoritma/
They are both translated articles (written originally by me on the same website). I've noticed those +1's (476 on both articles) when my website received a penalty for "SEO" keyword on Google.rs (Serbian Google) and I'm now on the 11th page.
Other keywords still rank just fine. Not cool, right? Now, I think there could be two solutions:
First one is to remove my inner link that's pointing to my homepage with "SEO" anchor, and hope for the best. Second one is to completely remove/delete those two articles and wait for Google to reindex the website and hopefully remove my ban. Do you guy have some other ideas how can I fix this or remove / disavow those +1 or somehow explain to the Google crew / algo that I'm just a humble SEO without any evil thoughts? 🙂 Thank you in advance.0