Link Juice passing through a redirect of a disallowed URL
-
Hey guys!
Suppose I disallow search bots from indexing anything on my secure server in my robots.txt, and 301 redirect all of my secure server traffic to my non-secure site. Will the search bots see the redirect before they realize that they're disallowed from accessing that page? Or will they see that page is disallowed and not follow the redirect?
Should I change my robots.txt to allow search bots to crawl my secure site so they can find the redirects?
-
Thanks Nick!
-
If you think there are any links out there that point to your secure pages and want to preserve any SEO related value from those links, you should remove the disallow from robots.txt and use 301 redirects. However, if you redirect your secure pages to non-secure, what is the point of having the secure pages? They would never be used since users would be 301 redirected to the non-secure address.
If you are trying to keep the secure pages out of the index to avoid duplicate content, and want users to be able to have access to the secure pages, disallow in robots.txt and remove the 301 redirects.
In the case of pages that don't really need to be secure, but are available both as http and https, you could also use the canonical tag to show either the secure or non-secure as the original. Individually redirecting the potentially duplicate pages rather than a sitewide https to http redirect would preserve any "juice" you may be getting from links to either version.
If it is more complicated, take a look at this thread on Googles support forum: http://www.google.com/support/forum/p/Webmasters/thread?tid=6a7fac6f81489bc2&hl=en
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Broken URL Links
Hi everyone, I have a question regarding broken URL links on my website. Late last year I move my site from an old platform to Shopify, and now have broken URL links giving out 4xx errors. When I look at Moz Pro>Campaigns>Insights>links, I can see the top broken URL links, however there is a difference if copy & paste URL directly from Moz Pro and by Export CSV file. For example below, If I copy and paste links direct from Moz Pro, it has the “http://” in front as below: http://www.thehairhub.com.au/WebRoot/ecshared01/Shops/thehairhub/57F3/1D8F/D244/C675/E27D/AC10/003F/35AD/manic-panic-colours.jpg But when I export the list of links as an CSV file, the http:// is removed. www.thehairhub.com.au/WebRoot/ecshared01/Shops/thehairhub/57F3/1D8F/D244/C675/E27D/AC10/003F/35AD/manic-panic-colours.jpg Another Example below: By copy & paste URL direct from Moz Pro
Technical SEO | | johnwall
http://thehairhub.com.au/Shop-Brands/Vitafive-CPR/CPR-Rescue By export CSV file.
thehairhub.com.au/Shop-Brands/Vitafive-CPR/CPR-Rescue Which one do I use to enter into the “Redirect From” field in Shopify URL Redirects? Do I need to have the http:// in front of the URL? Or is it not required for redirects to work? Kind Regards, John Wall
The Hair Hub0 -
Deleteing old page and passing on link strenth?
We are a printing company and thinking over bringing our products down to 2 - 3 rather than the 10+ we currently have, the pages we will be getting rid of will be pages such as flyers, booklets etc and just concentrating on banners and stickers would you suggest 301ing the pages to the home page or picking pages for them to go to? Also could we expect a decent raise for the pages we are left with? Thanks shaun
Technical SEO | | BobAnderson0 -
URL Structure
Hi, Hope you are all well. On our website we have a 'blog' and a 'news' section. The blog is located on "/blog" - but when you click on a post the url structure changes to /name-of-article and the blog subdomain isn't included. Would it be better to have "blog/name-of-article as this would then make the blog perform better in search results? Also, if our news page is under /news - but when you click on an article it changes to /news-article/name-of-article Wouldn't it be better to have /news/name-of-article Thanks a lot!! 🙂
Technical SEO | | National-Homebuyers0 -
Changing URLs
As of right now we are using yahoo small business, when creating a product you have to declare an id, when we created the site we were not aware that you will not be able to change the id but also the ID is being used as the URL. we have a couple thousand products in which we will need to update the URLs. What would the best way to be to fix this without losing much juice from our current pages. Also I was thinking that if we did them all in a couple weeks it would hurt us a lot, and the best course of action would be to do a slow roll out of the URL changes. Any help is appreciated. Thank you!
Technical SEO | | TITOJAX0 -
Backlink Profile: Should I disavow these links? Auto-Generated Links etc
Hello Moz Community, At first I wanted to say that I really like the Q&A section and that I read and learned a lot - and today it is time for my first own question 😉 I checked our backlink-profile these days and I found in my opinion a few bad/spammy links, most of them are auto-generated by pickung up some (meta) information from our webpage. Now my question is if I should dasavow these links over webmasters or if these links shouldn't matter as I guess basically every webpage will be picked up from them. Especially from the perspective that our rankings dropped significantly last weeks, but I am not sure if this can be the real reason. Examples are pages like: https://www.askives.com/ -Auto-Generates for example meta descriptions with links http://www.websitesalike.com/ -find similar websites http://mashrom.ir/ -no idea about this, really crazy Or we are at http://www.europages.com/, which makes sense for me and we get some referral traffic as well, but they auto-generated links from all their TLDs like .gr / .it / .cn etc. -just disavow all other TLDs than .com? Another example would be links from OM services like: seoprofiler.com Moreover we have a lot of links from different HR portals (including really many outdated job postings). Can these links “hurt” as well? Thanks a lot for your help! Greez Heiko
Technical SEO | | _Heiko_0 -
Proper way of handling wordpress urls and redirects?
I have recently changed some of my urls in wordpress + adding a sub category in the url structure. from
Technical SEO | | Livet
www.mydomain.se/category/subcategory to www.mydomain.se/category/subcategory/subcategory2/ My products are stil under www.mydomain.se/products/cool-product and are not affected. Should I 301 the old url (www.mydomain.se/category/subcategory) to the new (www.mydomain.se/category/subcategory/subcategory2/)? If that is so, can someone recommend a good 301 redirection plugin? Thanks!0 -
Could a URL change path conflict a 301 redirect?
Hi Mozzers, We create multiple pages for one of my client. Some of them are replacing old pages. I setup 5 of them out of 40. I was able to set them live via the drupal CMS. The new pages were actually published but didn't have any URL but had nodes in directory such as www.example.com/node298. To set them live i changed the url path to one page that already existed( www.example.com/old). In order to setup the replacing page: www.example.com/node298 i added the same name as the old one but in order to avoid URL conflicts with new page(www.example.com/new) I had to change the old page's url path as well such as www.example.com/old2) I know i have to 301 redirect the old to the new obviously but my question is: does a URL path change on the old page www.example.com/old matters in when 301 ing it? will it still transfer all the juice to the new page Visual Process: Main goal: www.example.com/old redirect to www.example.com/new but these two are exactly the same url So modification of URL path: www.example.com/old to www.example.com/old2 to avoid URL conflict Therefore www.example.com/old2 =www.example.com/old (just url change path difference) Question: Because of this url change, will a 301 from www.example.com/old2 to www.example.com/new will still carry all the juice that www.example.com/old carried or not? I hope i didn't make it too confusing. Let me know if it is the case Thanks Mozzers Ty
Technical SEO | | Ideas-Money-Art0 -
Not sure which URL to use for 301 redirect
A client has new website design completed by another developer, was launched in April of this year. No 301 redirect was set up so duplicate content is an issue. Client has had a website with same domain name for about 10 years, but has not had any SEO work completed before or since his new site design. For non-www there are 6 referring links - 1 considered to have authority, for www there are also 6 but 3 considered to have authority. More links seem to coming from www than non-www. But for one of the clients keywords they are ranked #1 for their area and that links to their non-www address. And even though no redirects set up by developer, non-www has had far more visits according to Google Analytics. So many basics that still need to be done for site: no meta-descriptions on any page, H1 and page titles could use keywords, call to action moved above fold, etc. Considering this is a new site, and new SEO work and many more inbound links needed, does it matter which address I redirect to? _Cindy Barnard
Technical SEO | | CeCeBar0