Redirected Subdomain Development URLs Showing In SERPs?
-
I develop client websites within a subdomain of another website (with noindex, nofollow so that incomplete websites on the wrong domains aren't ever seen by web users). Then, when we launch a client's site on their own domain, we redirect all of the development URLS to the appropriate page on the new live site. (meaning at site launch, all pages on http://client-site.developersite.com would be set to 301 redirect to identical pages pages on http://www.client-site.com).
This system has always seemed to work fine, but today I discovered 94,700 pages indexed by Google on my root domain and found that these were mostly old URLs of sites in development that redirect to the actual client sites. Many are several years old. Any idea why Google would be indexing these pages?
Thanks in advance!
-
Agreeing with Brent on password-protection. And everything else he said.
Also, if these are CMS-driven sites, I can almost guarantee you you've got paths pointing back to the dev-version. Probably absolute links to images and such. I've had that happen before. Remove the database and files from your dev directories and see what breaks on the live-site
-
I guess my question is why would you have to 301 redirect from your development (staging) site to the client's live site if you had nofollow, noindex? Google should not know about the dev site, or even be able to access it.
Sounds like in the new client site there are links going back to the development site. (maybe rel=canonical). Or there is a link out there linking to content you don't want found.
Google's not going to index it unless it finds it. Just jump into Google Webmaster and remove those directories.
Remember, noindex,nofollow or robots.txt are not 100% answers. You should make these sites password protected.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
WP URL issue - Concatenated URLs (LOTS of them)
WP is doing this somehow, and creating URLs for hundreds of pages that don't exist. HOW is this happening, and how do I stop It? I have many, many URLS like this: https://www.atouchofrust.com/terms-of-use/atouchofrust.com/vendor-news. Of note, atouchofrust.com/terms-of-use, and atouchofrust.com/vendor-news are both legit pages on the site. Why they are being concatenated is beyond my limited understanding of WP. Please, somebody, help. Cori
Technical SEO | | FlyingC0 -
URL structure change for pages without traffic: 301 redirect or not ?
Hi, I am just starting with MOZ PRO and trying to handle the high priority issues, starting with pages with 4XX Client Error. I am wondering what we should do with pages with no traffic and no external links. For instance: So time ago we change the URL structure of our blog to a flatter one, and so eg we moved a page: from: domain-name/dla-rodzicow/poradniki/poradniki-po-markach/vilac/vilac-zabawki-z-dusza to: domain-name/dla-rodzicow/poradniki/marka-vilac/vilac-zabawki-z-dusza/ Still not very flat but this is not the point. MOZ PRO shows we are having internal links to the old url. According to MOZ PRO, we don't have external links. According to Analytics we have no traffic on the old page. So now we changed the internal link, and I am wondering whether we should 301 redirect the old page to the new one, or whether a sitemap update is enough for this kind of pages ? Thanks in advance for your help.
Technical SEO | | isabelledylag0 -
Pages with 301 redirects showing as 200 when crawled using RogerBot
Hi guys, I recently did an audit for a client and ran a crawl on the site using RogerBot. We quickly noticed that all but one page was showing as status code 200, but we knew that there were a lot of 301 redirects in place. When our developers checked it, they saw the pages as 301s, as did the Moz toolbar. If page A redirected to page B, our developers and the Moz toolbar saw page A as 301 and page B as 200. However the crawl showed both page A and page B as 200. Does anyone have any idea why the crawl may have been showing the status codes as 200? We've checked and the redirect is definitely in place for the user, but our worry is that there could be an issue with duplicate content if a crawler isn't picking up on the 301 redirect. Thanks!
Technical SEO | | Welford-Media0 -
Redirects
I have a question about 404ed domains and old domains. #1 A domain has many links to it, but has been 404ed for 4 months. Should I redirect to a page I own and is almost exactly the same content. Will the fact that it was once 404ed be an issue? #2 I have an old domain that has many links but has been stagnant for a long time. Are these links still valuable and I should I redirect them to an important page on a different site? Does penguin influence your advice?
Technical SEO | | tylerfraser0 -
Delete 301 redirected pages from server after redirect is in place?
Should I remove the redirected old pages from my site after the redirects are in place? Google is hating the redirects and we have tanked. I did over 50 redirects this week, consolidating content and making one great page our of 3-10 pages with very little content per page. But the old pages are still visible to google's bot. Also, I have not put a rel canonical to itself on the new pages. Is that necessary? Thanks! Jean
Technical SEO | | JeanYates0 -
Geo-Redirection
Our client has two almost identical sites targeting: Australia (www.mysite.com.au) Rest of World (www.mysite.com) Currently they have splash page on www.mysite.com asking users to select: Australia Rest of World (redirects to: www.mysite.com/home) I'm thinking they should get rid of slash page and simply auto detect if user on www.mysite.com is based in Australia and serve a message "Do you want to visit our .com.au site. It's not helped by the fact the .com site appears to get served ahead of .com.au iin australia as both sites are hosted in US. Looking to change this! Thanks in advance for your help!
Technical SEO | | steermoz70 -
Disallowing https URLs
It there a problem disallowing all https URLs to be indexed in order to avoid duplication? This is the article recommending this practice - http://blog.leonardchallis.com/seo/serve-a-different-robots-txt-for-https/ Thanks!
Technical SEO | | theLotter0 -
Why is this url showing as "not crawled" on opensiteexplorer, but still showing up in Google's index?
The below url is showing up as "not crawled" on opensitexplorer.com, but when you google the title tag "Joel Roberts, Our Family Doctors - Doctor in Clearwater, FL" it is showing up in the Google index. Can you explain why this is happening? Thank you http://doctor.webmd.com/physician_finder/profile.aspx?sponsor=core&pid=14ef09dd-e216-4369-99d3-460aa3c4f1ce
Technical SEO | | nicole.healthline0