Removing secure subdomain from google index
-
we've noticed over the last few months that Google is not honoring our main website's robots.txt file. We have added rules to disallow secure pages such as:
Disallow: /login.cgis Disallow: /logout.cgis Disallow: /password.cgis Disallow: /customer/* We have noticed that google is crawling these secure pages and then duplicating our complete ecommerce website across our secure subdomain in the google index (duplicate content) https://secure.domain.com/etc. Our webmaster recently implemented a specific robots.txt file for the secure subdomain disallow all however, these duplicated secure pages remain in the index.
User-agent: *
Disallow: /My question is should i request Google to remove these secure urls through Google Webmaster Tools? If so, is there any potential risk to my main ecommerce website? We have 8,700 pages currently indexed into google and would not want to risk any ill effects to our website. How would I submit this request in the URL Removal tools specifically? would inputting https://secure.domain.com/ cover all of the urls? We do not want any secure pages being indexed to the index and all secure pages are served on the secure.domain example. Please private message me for specific details if you'd like to see an example. Thank you,
-
I think you're saying you have
mainwebsitethatsellsstuff.com
securesubdomainof.mainwebsitethatsellsstuff.comand that you want to keep the main domain, and remove the subdomain, and that it's not a case of http vs https with the URL otherwise being the same, right?
You can verify a subdomain in Google Webmaster Tools and remove the entire subdomain. I've had to do this for a dev subdomain that accidentally got indexed. I was able to keep the main domain, and remove the subdomain. The key is to verify that subdomain, and leave the main domain alone, provided I'm understanding your question correctly.
-
Do you need 8700 pages served on https? Protocol should transition when a page is ok to serve unsecured. Generally you would only serve pages on https that contain confidential information and have general content on http. If you look at the site and ask how many of those pages can a no logged in user see? If they are not protected by authorization then they do not need https as the content is publically viewable.
-
URL Removal would not be a good action in this case. According to Google, when they remove the https version, they will also remove the http version along with it.
How long ago did you implement the robots.txt exclusion for the https pages? It will take Google some time to pull this from their index. To help you can add the following on your https pages which will keep the pages from continuing to be cached:
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How Google's "Temporarily remove URLs" in search console works?
Hi, We have created new sub-domain with new content which we want to highlight for users. But our old content from different sub-domain is making top on google results with reputation. How can we highlight new content and suppress old sub-domain in results? Many pages have related title tags and other information in similar. We are planing to hide URLs from Google search console, so slowly new pages will attain the traffic. How does it works?
Algorithm Updates | | vtmoz0 -
Google panda, penguin or Patience needed?
Dear friends, On 3rd of May, i suffered a Manual google unnatural outbound link penalty. I recovered from the said penalty on 27th of June. However, i have noticed that my traffic has been dropping since 23rd April. I am confused where to target my work. Should i work on thin content and is it an algorithmic Panda problem (but my keywords are still ranking good) or is it a Penguin problem (I had 6 domains with payday loans backlinks and i have dosavowed 32 backlinks recently) What should be my plan of action here and what would you recommend? An image is attached herewith for your reference, PHd8BzX.png
Algorithm Updates | | marketing911 -
If Google Trends Doubles?
If google shows a search trend doubling in a time frame, does that mean the amount of searches doubled? As in: 2006 was ranked at a 50 on trends and the 100 is 2013 and in 2013 10,000 searches were made, does that mean around 5,000 searches were made in 2006?
Algorithm Updates | | JoshBowers20120 -
I need help with drastic SERP difference between Bing and Google
One of our sites that has been around for a couple of years has about 60,000 pages showing on google, however, bing only shows 90 pages for the site. This same phenomenon has been happening across the board for our sites. Any ideas to improve our indexing results for bing?
Algorithm Updates | | atuomala0 -
You folks see the Google Analytics Real Time View (Beta)
I was pretty impressed by the amount of data they can present to you and how quickly the site gets it to you. Hit your GA account click the Home tab then click the Real Time (Beta) link in the navigation. It is pretty impressive. GoogleAnalytics.jpg
Algorithm Updates | | donford0 -
Rankings changing every couple of MINUTES in Google?
We've been experiencing some unusual behaviour in the Google.co.uk SERPs recently... Basically, the ranking of some of our websites for certain keywords appears to be changing by the minute. For example, doing a search for "our keyword" might show us at #20. Then a few minutes later, doing the same search shows us at #14, and then the same search a few minutes later shows us at #26, and then sometimes we're not ranked at all, etc etc. I know the algorithm changes a lot, but does it really change every couple of minutes? Has anyone else experienced this kind of behaviour in the SERPs? What could be causing it to happen?
Algorithm Updates | | d4online0 -
Any ideas why our category pages got de-indexed?
Hi all, I work for evenues, a directory website that provides listings of meeting rooms and event spaces. Things seemed to be chugging along nicely with our link building effort (mostly through guest blogging using a variety of anchor text). Woke up on Monday morning to find that our City pages have been de-indexed. This page: http://www.evenues.com/Meeting-Spaces/Seattle/Washington used to be at the top of page #2 in the SERPs for the keyword "Meeting Rooms in Seattle" I doubt that we got de-indexed because of our link building efforts, as it was only a few blog posts and links from profile pages on community websites. My guess is that when we did a recent 2.0 release of the site, there are now several "filters" or subcategory pages with latitude and longitude parameters in the URL + different page titles based on the categories like: "Meeting Rooms and Event Spaces in Seattle" --Main Page "Meeting Rooms in Seattle" "Classroom Venues in Seattle" "Party Venues in Seattle" There was a bit of pushback when I suggested that we do a rel="canonical" on these babies because ideally we'd like to rank for all 4 queries (Meeting Rooms, Party Venues, Classrooms, in City). These are new changes, and I have a sneaking suspicion this is why we got de-indexed. We're presenting generally the same content. Thoughts?
Algorithm Updates | | eVenuesSEO0 -
What is the critical size to reach for a content farm to be under google spot?
We're looking for building a content farm, as an igniter for another site, so there will be some duplicate content. Is it a good or a bad strategy in terms of SEO.
Algorithm Updates | | sarenausa0