Removing secure subdomain from google index
-
we've noticed over the last few months that Google is not honoring our main website's robots.txt file. We have added rules to disallow secure pages such as:
Disallow: /login.cgis Disallow: /logout.cgis Disallow: /password.cgis Disallow: /customer/* We have noticed that google is crawling these secure pages and then duplicating our complete ecommerce website across our secure subdomain in the google index (duplicate content) https://secure.domain.com/etc. Our webmaster recently implemented a specific robots.txt file for the secure subdomain disallow all however, these duplicated secure pages remain in the index.
User-agent: *
Disallow: /My question is should i request Google to remove these secure urls through Google Webmaster Tools? If so, is there any potential risk to my main ecommerce website? We have 8,700 pages currently indexed into google and would not want to risk any ill effects to our website. How would I submit this request in the URL Removal tools specifically? would inputting https://secure.domain.com/ cover all of the urls? We do not want any secure pages being indexed to the index and all secure pages are served on the secure.domain example. Please private message me for specific details if you'd like to see an example. Thank you,
-
I think you're saying you have
mainwebsitethatsellsstuff.com
securesubdomainof.mainwebsitethatsellsstuff.comand that you want to keep the main domain, and remove the subdomain, and that it's not a case of http vs https with the URL otherwise being the same, right?
You can verify a subdomain in Google Webmaster Tools and remove the entire subdomain. I've had to do this for a dev subdomain that accidentally got indexed. I was able to keep the main domain, and remove the subdomain. The key is to verify that subdomain, and leave the main domain alone, provided I'm understanding your question correctly.
-
Do you need 8700 pages served on https? Protocol should transition when a page is ok to serve unsecured. Generally you would only serve pages on https that contain confidential information and have general content on http. If you look at the site and ask how many of those pages can a no logged in user see? If they are not protected by authorization then they do not need https as the content is publically viewable.
-
URL Removal would not be a good action in this case. According to Google, when they remove the https version, they will also remove the http version along with it.
How long ago did you implement the robots.txt exclusion for the https pages? It will take Google some time to pull this from their index. To help you can add the following on your https pages which will keep the pages from continuing to be cached:
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Site appearing and disappearing from google serps.
Hi, My website is normally on page 2-3 on google consistently. Over the past month it has been appearing and then completely disappearing from the serps. One day it will be on page 2, then the next day completely missing from the serps. When i check the index it seems to be indexed correctly when doing site:mysite.com. I don't understand why this keeps happening, any experience with this issue? It doesn't seem to be a google dance as far as I can tell. When my other sites dance they typically just go up or down a few ranks for a couple weeks until they stabilize. Not completely fall off the search engine.
Algorithm Updates | | Chris_www0 -
Google ranking penalty: Limited to specific pages or complete website?
Hi all, Let's say few pages on the website dropped in the rankings due to poor optimisation of the pages or hit by algo updates. Does Google limits the ranking drop only to these pages or the entire website will have any impact? I mean will this cause ranking drop to the homepage for primary keyword? Will Google pose the penalty to other pages in the website if few pages drop in the rankings. Thanks
Algorithm Updates | | vtmoz0 -
Training - How Google Crawls & Indexes Websites
Hi Does anyone know of any online training resources/webinars/training UK based that will cover the following for SEO: Why monitoring how search engines crawl and index content is important and how this can improve your SEO performance Using Google advanced operators to evaluate website indexation How to use log file data to gain insight into how search engines crawl and index content Techniques to control how search engines crawl and index content How search engines deal with JavaScript, common frameworks and SEO considerations I'm trying to develop my technical knowledge - I have always been more focused on content/KWD research/optimisation. Thank you
Algorithm Updates | | BeckyKey0 -
SEO - Google Local Listing & Same Day Delivery
Hi We are looking to offer same day delivery if you're in a 20 mile radius to us. I'm trying to do some research on how to optimise this for Google organic listings. Would this be the same as optimising for a local business listing? I'm not sure where to start. Thanks! Becky
Algorithm Updates | | BeckyKey0 -
Does Google use dateModified or date Published in its SERPs?
I was curious as to the prioritization of dateCreated / datePublished and dateModified in our microdata and how it affects google search results. I have read some entries online that say Google prioritizes dateModified in SERPs, but others that claim they prioritize datePublished or dateCreated. Do you know (or could you point me to some resources) as to whether Google uses dateModified or date Published in its SERPs? Thanks!
Algorithm Updates | | Parse.ly0 -
What are the top tips for winning on Google, Bing, and Yahoo?
We just launched a new site that is starting to be indexed well in Google, but Bing and Yahoo are lagging a bit. I understand that the search engines are different algorithms and will take different lengths of time to index, rank, etc. What I'm curious about is are there any other tips / advice / things to keep in mind when trying to rank on the different search engines? Thanks!!
Algorithm Updates | | Emily_A0 -
Am I the only one experiencing this Google SERP problem?
I perform Google searches every single day, sometimes several times in a day. These searches have nothing to do with being a marketer--they're simply as a consumer, researcher, person who needs a question answered, or in other words: a typical person. For about the past month or so, I have been unsuccessful at finding what I'm looking for on the first try EVERY SINGLE TIME. Yes, I mean it--every single time. I'm left either going all the way to the third page, clicking dozens of results and retuning to the SERPs, or having to start over with a differently worded query. This is far too often to be a coincidence. Has this been happening to anymore else? I know there was a recent significant algorithm update, right? I always look at algorithm updates through the eyes of an SEO, but I'm currently looking at it through the eyes for an average searcher, and I'm frustrated! It's been like trying to find something on Bing!
Algorithm Updates | | UnderRugSwept0 -
Can Google display a diffrent page title?
Hi if I search google UK for the phrase car leasing, google returns my listing as Car Lease Deals However the same search on Yahoo or Bing bring back Contract Hire | Vehicle & Car Leasing Deals | Car Lease Deals this is the real page title. Why would this happen? Thanks Andy
Algorithm Updates | | First-VehicleLeasing0