Ah ok. I understand now. I wasn't picking up on what you were saying before.

If with the soft launch you are already putting the "new" version of the site on their intended final URLs then yes, you can let the engines start crawling those URLs. For each new URL you let the search engines crawl make sure to 301 its corresponding old URL (the old site) to the new version to minimize any duplicate content issues.

If for whatever reason you can't quite 301 the old URLs yet (like if you still need instant access to reroute traffic back to them) you could try using rel=canonical on the old pages and point them to their new counter part only if the main content on each of the pages is almost exactly the same. You don't want Google to think you're manipulating them with rel=canonical.

As I'd said, there really isn't a reason to let them get a head start. The URL's will be changing when you transition the new site out of the subdomain (ie beta.sierratradingpost.com/mens vs sierratradingpost.com/mens - those are considered 2 completely different URLs) and the engines will have to recrawl all of the new pages at that point anyway.

And when you drop the sub domain you definitely want to 301 all of the old site structure's  URLs to their corresponding new page's URLs. That way nothing gets lost in the transition.

Yeah, just refer to our conversation above as I think it will pertain better to your situation.

The only issue is that you have to keep in mind that Google/Bing defines pages on the internet through their URL's, not the content. The content only describes the pages.

So if you let the engines pre crawl the pages before dropping the subdomain - simply for the reason of letting them have a "sneak peek" - you won't really be doing yourself much of a favor, as the engines will just be recrawling the content on the non subdomain URL as if it were brand new anyway.

The reason to do it the pre crawl way would be if you're already building back links to the new beta pages. Then it could make sense to let the engines index those pages and 301 them to their new non subdomain versions later. In my opinion the benefit from this route would outweigh any potential duplicate content issues.

What YesBaby is talking about is somehting like Google's Website Optimizer. When someone goes to sierratradingpost.com/mens-stuff, for example, it will give 50% of the people the old version of the site for that page, and the other 50% the new version. It will eliminate any duplicate content issues as the 2 page variations will still be attached to the same exact URL.

Definitely a viable option if it fits with your game plan of how you want to do things.

First of all - I love the new design. It looks great!

The absolutel best way to go about it in my opinion would be to simply have the new site ready, and then launch it fully under the base domain (no subdomain) while 301 redirecting important old pages on the site to their related new versions. That way the search engine will have the easiest time of discovering the new site and indexing it, while making sure you don't lose anything in the transition via proper 301'ing.

I can't say it would provide you with a massive benefit to set up a way for the search engines to start crawling the new site for now, as you're just going to be moving all of those URL's off of the subdomain in the near future anyway - where they will then need to be recrawled on the parent domain as if they were brand new.

If the traffic diverter you have set up automatically 301's requests for old site pages to their new beta URL version then you might as well let those new versions be indexed for the time being. Just make sure that when you transfer the beta site to the parent domain that you 301 the old beta URL's to their new permanent home.

Let me make sure I have this straight... you're not going to be directing the new site format to a subdomain permanently, right? You were only using the sub domain for beta purposes?

The way I see it, when I go to Sierra Trading Post's site now I can make out what looks like 2 different types of architecture structures. You have one link on the page pointing to Men's clothing which executes at a single defined .htm file. Then you can see that you have the "Men's Classics" (still general men's clothing?) which points to a directory which I'm guessing is your new site. Correct me if I'm wrong on this, or if I'm right but have the old vs. new reversed.

If that is the case your best bet to try and minimalize any ranking impact would be to 301 redirect pages from the old catalog architecture to the new. That way you could remove the old site files completely and let the server take care of the direction.

If you need to leave the old site up for throttling purposes like you said - you could use canoniclazation tags to refer the old pages to the new ones. That along with employing 301 tags would help train the search engines into understanding what you're doing.

I'm sorry if I didn't answer your question as you needed. I'm still not sure if I understood your issue as intended. =P

I understand. In that case as I stated in my edit I don't think most of the "well known" directories of that sort have a whole lot of their link value passed through by Google. I would say the money could definitely be invested in another tactic that will provide a better ROI, but that is of course an opinion which could be incorrect.

Well the problems you're trying to overcome are the exact reasons why a good CMS blog system pulling and storing posts from a databse is extremely effective.

Doing things your way, all static HTML/CSS with no databse, it would definitely make sense to only list the most recent posts on any given page/chategory, and then come up with an archive system for the rest.

You should have a search feature you can put on your site so as to let people easily pull up older buried posts. I don't personally have any experience with it, but you could try Google's Custom Search Engine to see if it could accomplish what you need.

As far as the hierarchy of the domain levels goes I would never go deeper than 4 levels with your categories/posts. You can almost never have too flat of a hierarchy... example being looking at SEOMoz's structure. They are a massive blog with a large number of posts, and yet almost all of the posts trace to a URL structure of seomoz.org/blog/post-name. So in theory that would create a very broad and flat structure, yet they don't seem to have much if any indexing issues.

On many of our client sites we've actually seen positive impacts from the farm updates.

Previously, some of our clients were being beat out for the top few positions by some content farms for various long tail search terms.

After the Farm Update we've seen those content farm page results drop off into oblivion and our client sites have stepped right up into their positions. Our client's have gained dozens of new 1st, 2nd, and 3rd position SERP results on long tail keywords.

So in summary - none of the websites we manage/seo have been hurt by the Farmer Update. We have, instead, been rewarded from it.

Hi David,

First of all you need to keep in mind that paid directories may be able to take your money, but they can never take YOUR FREEEDOOMMMMM!!!!!

Now that I've got that out of the way (and hopefully delivered a mild chuckle at least) on to the business at hand.

There is a fundamental problem with "paid" directories. That being they have 2 options when it comes to the links they provide from their directory listings. They either need to 'nofollow' the links to not have Google penalize them as a link buying service, or they need to hide the fact that they are indeed basically just a link buying service... otherwise Google will likely penalize them.

You need to look at a paid directory from Google's point of view. If the directory is openly advertising that you can pay them for inclusion, and thus get a link from their site for the money then Google has just about no choice but to flag the directory as a link buying service.

If the directory already links to your site for free as a service to their customers, Google can consider them a legitimate service and not penalize the directory. Many directories of course do this and then offer premium type services that business can pay for and not receive any Google penalty from (yelp and even the BBB being great examples). In this case, though, you're not really receiving any extra "link juice" for the service you're buying. You're usually just receiving extra visibility or an endorsement of some sort.

So in short, if you're trying to find a paid directory for the purposes of acquiring links I would strongly suggest you rethink your strategy. If you are, however, simply looking to "buy links" then disregard this advice.

EDIT - Something I realized I need to add. You stated you do recognize that BotW rarely shows up in back link reports. Site's such as BotW and others (JoeAnt and so on) I think don't pass much link juice through to websites from Google and Bing. The search engines recognize that these sites offer paid review services, and while it's certainly not like link buying because instead there are editors supposedly quality checking the URL's submitted, I would guess the search engines still do not pass full link value on to websites.

I could be very wrong of course - I've just never seen any noticeable impact from going through directories such as those.