Link Juice passing through a redirect of a disallowed URL
-
Hey guys!
Suppose I disallow search bots from indexing anything on my secure server in my robots.txt, and 301 redirect all of my secure server traffic to my non-secure site. Will the search bots see the redirect before they realize that they're disallowed from accessing that page? Or will they see that page is disallowed and not follow the redirect?
Should I change my robots.txt to allow search bots to crawl my secure site so they can find the redirects?
-
Thanks Nick!
-
If you think there are any links out there that point to your secure pages and want to preserve any SEO related value from those links, you should remove the disallow from robots.txt and use 301 redirects. However, if you redirect your secure pages to non-secure, what is the point of having the secure pages? They would never be used since users would be 301 redirected to the non-secure address.
If you are trying to keep the secure pages out of the index to avoid duplicate content, and want users to be able to have access to the secure pages, disallow in robots.txt and remove the 301 redirects.
In the case of pages that don't really need to be secure, but are available both as http and https, you could also use the canonical tag to show either the secure or non-secure as the original. Individually redirecting the potentially duplicate pages rather than a sitewide https to http redirect would preserve any "juice" you may be getting from links to either version.
If it is more complicated, take a look at this thread on Googles support forum: http://www.google.com/support/forum/p/Webmasters/thread?tid=6a7fac6f81489bc2&hl=en
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Resolving 301 Redirect Chains from Different URL Versions (http, https, www, non-www)
Hi all, Our website has undergone both a redesign (with new URLs) and a migration to HTTPS in recent years. I'm having difficulties ensuring all URLs redirect to the correct version all the while preventing redirect chains. Right now everything is redirecting to the correct version but it usually takes up to two redirects to make this happen. See below for an example. How do I go about addressing this, or is this not even something I should concern myself with? Redirects (2) <colgroup><col width="123"><col width="302"></colgroup>
Technical SEO | | theyoungfirm
| Redirect Type | URL |
| | http://www.theyoungfirm.com/blog/2009/index.html 301 | https://theyoungfirm.com/blog/2009/index.html 301 | https://theyoungfirm.com/blog/ | This code below was what we added to our htaccess file. Prior to adding this, the various subdomain versions (www, non-www, http, etc.) were not redirecting properly. But ever since we added it, it's now created these additional URLs (see bolded URL above) as a middle step before resolving to the correct URL. RewriteEngine on RewriteCond %{HTTP_HOST} ^www.(.*)$ [NC] RewriteRule ^(.*)$ https://%1/$1 [R=301,L] RewriteCond %{HTTPS} !on RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L] Your feedback is much appreciated. Thanks in advance for your help. Sincerely, Bethany0 -
Purchased domain with links - redirect page by page or entire domain?
Hi, I purchased an old domain with a lot of links that I'm redirecting to my site. I want all of their links to redirect to the same page on my site so I can approach this two different ways: Entire site
Technical SEO | | ninel_P
1.) RedirectMatch 301 ^(.*)$ http://www.xyz.com or Page by page
2). Redirect 301 /retiredpage.html http://www.xyz.com/newpage.html Is there a better option I should go with in regards to SEO effectiveness? Thanks in advance!0 -
Friendly URLS (SEO urls)
Hello, I own a eCommerce site with more than 5k of products, urls of products are : www.site.com/index.php?route=product/product&path=61_87&product_id=266 Im thinking about make it friend to seo site.com/category/product-brand Here is my question,will I lost ranks for make that change? Its very important to me know it Thank you very much!
Technical SEO | | matiw0 -
Why are pages linked with URL parameters showing up as separate pages with duplicate content?
Only one page exists . . . Yet I link to the page with different URL parameters for tracking purposes and for some reason it is showing up as a separate page with duplicate content . . . Help? rpcIZ.png
Technical SEO | | BlueLinkERP0 -
Right redirect to transfer juice www, no-www and website movement
Hi here is my problem, I have a website which works both with www. and without www; Now I want to optimize it and transfer it to another new domain along with the link juice of both the old domain with www and without www. I don't want to do it with multiple redirects but with just one clean redirect (I know that search engines don't like that, am I right?). What should I write in the .htaccess of the old domain and in the one of the new domain? To summarize I want something like that: www.oldsite.com/anyfile.html>301>www.newsite.com/anyfile.html oldsite.com/anyfile.html>301>www.newsite.com/anyfile.html newsite.com/anyfile.html>301?>www.newsite.com/anyfile.html Thanks
Technical SEO | | darkanweb0 -
Paid outbound links - do they have juice?
Hi, A client is asking about a website that has a business model I don't normally work with - it's a site full of very informative articles - no harm there - and they are sponsored by companies who provide the service being discussed. Not really any harm there. But attached to the link is an outbound tracking device. The business model is pay per click. Has anyone had experience with this, specifically if larger search engines have picked up on this and filtered out those types of links to not count very much if at all? Thanks!
Technical SEO | | lilactree0 -
Should we block URL param in Webmaster tools after URL migration?
Hi, We have just released a new version of our website that now has a human readable nice URL's. Our old ugly URL's are still accessible and cannot be blocked/redirected. These old URL's use a URL param that has an xpath like expression language to define the location in our catalog. We have about 2 million pages indexed with this old URL param in it while we have approximately 70k nice URL's after the migration. This high number of old URL's is due to facetting that was done using this URL param. I wonder if we should now completely block this URL param from Google Webmaster tools so that these ugly URL's will be removed from the Google index. Or will this harm our position in Google? Thanks, Chris
Technical SEO | | eCommerceSEO0 -
URL Structure
Hi Guys, I'm in the process of creating a very exciting startup aimed at the baby industry. It's essentially a social commerce question where parents can shop for products, create lists of products and ask questions. The challenge I'm facing is how best to structure my URLs from an SEO standpoint. For example a common baby topic such as "feeding", can sit in all three categories: Shopping category aggregates all products related to feeding List category aggregates all lists related to feeding Question category aggregates all question and answers on feeding So for that keyword "feeding" you have 3 potential landing pages. What I was wondering is what is the most effective way of doing it? I was thinking of something along these lines: /shopping/feeding /baby_list/feeding /ask/feeding Would love to hear your points of view on this. Thanks! Walid
Technical SEO | | walidalsaqqaf0