Site being indexed by Google before it has launched
-
We are currently coming towards the end of migrating one of our retail sites over to magento. To our horror, we find out today that some pages are already being indexed by Google, and we have started receiving orders through new site. Do you have any suggestions for what may have caused this? Or similarly, what the best solution would be to de-index ourselves? We most recently excluded anything with a certain parameter from robots.txt - could this being implemented incorrectly have caused this issue?
Thanks
-
Google has ignored a robots.txt for my site because it thought we were "hiding" important content.
Depending on your site structure, putting a "no index" tag on your page is also a good idea. Doing that as pages are built can prevent it (better) from being indexed in the first place. Then you remove the tag when you are ready to go live. And, submit your updated sitemap via GWT when you finish your move so Google will know what pages to index.
-
I would be very careful on how you manage this right now. It all depends on the old and new urls. If this is something in Dev staging, like http://dev.examplesite.com, then it is alright to do everything possible to clean them up. But if the new urls are going to be the new final urls, then you have to be careful.
The best thing you could do right now would be to password protect the new site. That way there isn't a chance of anything getting to it.
All it takes is Google finding one link to the site then it will crawl from there.
-
Exclude the whole new site using robots.txt, and remove the current new site-indexed URL's within Webmaster Tools: Optimization > Remove URL's.
When the new site is ready to launch, undo these removed URL's using the "Reinclude" button, also found within the Remove URL's section of Webmaster Tools. Do this prior to 301'ing the old site pages to the new, and you will probably wish to announce the new site launch via social media as well to encourage indexation. +1'ing a page is considered the fastest way to get it indexed.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
No Longer Indexed in Google (Says Redirected)
Just recently my page, http:/www./waikoloavacationrentals.com/mauna-lani-terrace, was no longer indexed by google. The sub pages from it still are. I have not done anything sketchy with the page. When I went into the google fetch it says that it is redirected. Any ideas what is this all about? Here is what it says for the fetch: Http/1.1 301 moved permanently
Technical SEO | | RobDalton
Server: nginx
Date: Tue, 07 Mar 2017 00:43:26GMT
Content-Type: text/html
Content-Length: 178
Connection: keep-alive
Keep-Alive: timeout=20
Location: http://waikoloavacationrentals.com/mauna-lani-terrace <title>301 moved permanently</title> <center> 301 moved permanently </center> <center>nginx</center>0 -
How can I tell Google not to index a portion of a webpage?
I'm working with an ecommerce site that has many product descriptions for various brands that are important to have but are all straight duplicates. I'm looking for some type of tag tht can be implemented to prevent Google from seeing these as duplicates while still allowing the page to rank in the index. I thought I had found it with Googleoff, googleon tag but it appears that this is only used with the google appliance hardware.
Technical SEO | | bradwayland0 -
SEO across sites built using Google Web Toolkit
Hi guys, General question around general SEO best practices, such as url and title, and how they fit in with Google Web Toolkit built sites that use a www.site.com/#!category=12345 format. The space we're getting into is heavily competitive, with many established players doing standard SEO well. I know there are some speed benefits to using GWT, however I'd like to better understand the SEO impact, if any, before the site development progresses too far. Cheers, Jez
Technical SEO | | jez0000 -
What needs to be done to tell google my site has moved /changed
Hi everyone, I have a site, which I have re-built on a temporary domain, so that my main ecommerce site can still run.. I have noticed that google has already crawled my temporary domain. The only problem is I now want to transfer the new site back onto its proper domain (www.ourbrand.com). I have changed some of the URL structures of the new site so realize I will need to do re-directs relating to the same domain, but will google get confused that another domain used to have my new website on? I don't plan on using the old temporary domain again and wondered if I need to tell google in some way it was used just to build my site on? Michelle
Technical SEO | | nutjobshell0 -
Google penalising my site? Ranking now outside top 50
Hey there, Was wondering if anyone could give me some advice/help on my current problems with my Google ranking. I used to be on page one this time last year (still am on yahoo and bing) but Google has pushed me out of the top 50. Since then I built a new site on wordpress with new content but it hasn't made any real difference. This is my site address: www.elvisimpersonator4u.co.uk I also have a separate blog here: www.thisiselvis.tv The blog did have some duplicate writing on it which I removed a few months back but it still hasn't improved my Google ranking. Would a 301 redirect fix this problem properly? I've done quite a lot of SEO on several pages on the new site to try and improve it but no luck so far. In fact I hope I haven't over optimised! No tools seem to indicate this though. I have no messages in webmaster tools so there's no obvious penalty, my web hosts keep saying I just need to buy more link builds and content writing from them but I've already done a lot of this and it's made no discernible difference. If someone could have a look and give me some advice I would be most grateful! Thanks,
Technical SEO | | devrox
Jim0 -
Unnecessary pages getting indexed in Google for my blog
I have a blog dapazze.com and I am suffering from a problem for a long time. I found out that Google have indexed hundreds of replytocom links and images attachment pages for my blog. I had to remove these pages manually using the URL removal tool. I had used "Disallow: ?replytocom" in my robots.txt, but Google disobeyed it. After that, I removed the parameter from my blog completely using the SEO by Yoast plugin. But now I see that Google has again started indexing these links even after they are not present in my blog (I use #comment). Google have also indexed many of my admin and plugin pages, whereas they are disallowed in my robots.txt file. Have a look at my robots.txt file here: http://dapazze.com/robots.txt Please help me out to solve this problem permanently?
Technical SEO | | rahulchowdhury0 -
Why did our site drop in Google rankings?
My site's URL (web address) is: http://tinyurl.com/3svn2l9 Hi there, We operate a travel site that lists numerous tours, accommodation and activities. Since 6th August 2011 we have dropped from top 10 SERP rankings of our pages to around result number 100 (page 10) and losing massive amount of visitors via Google Search. Our Yahoo and Bing rankings are still in the top10. We need your advice and quick! The last changes we have made are the following: -redirected the non-www version to the www version on the 1st August -bought advertising with a follow link in a sidebar that is being populated across the site (+4000 pages) about 2 months ago -added a blog to the website 2 weeks ago and posted 2 posts to date. Additionally, our website structure allows visitors (and bots) to see the same listings via different URLs which caused duplicate content. This has been the case since the launch of our website about 1 year ago. To prevent this duplicate content we have placed canonical tags on the individual listings pages. Why did our site all of a sudden plummet in the rankings?
Technical SEO | | Robbern0 -
How to remove a sub domain from Google Index!
Hello, I have a website having many subdomains having same copy of content i think its harming my SEO for that site since abc and xyz sub domains do have same contents. Thus i require to know i have already deleted required subdomain DNS RECORDS now how to have those pages removed from Google index as well ? The DNS Records no more exists for those subdomains already.
Technical SEO | | anand20100