trung.ngo - check out this article I posted http://www.blindfiveyearold.com/crawl-optimization

that's where I got my "inspiration" from to consider using robots.txt instead...

I am thinking if I exclude more thin pages from being crawled (robots.txt) that may be better than my current "noindex, follow" - the thin pages are already "noindex, follow".

You are saying "unless there's evidence that the pages are taking up too much of the crawl bandwidth, it doesn't seem like too much of an issue to me." - but how would I know this? Fair to assume for a website with 5,000 pages this is probably not an issue?

I am concerned with the "noindex, follow" Google may think "ahh, we have seen all this stuff before. Thanks for keeping out of our index, but we are still going to devalue your original content indexed pages because we crawl and see all this thin stuff." I am thinking with the robots.txt it would potentially be a stronger signal that could help my indexed pages. Or you think it is a minor and probably not relevant?

Hi Keri, There are some good comments but none really answer this question and that is why I am trying to approach from different angles. Maybe you can shed some light on this:
AJ Kohn wrote this great article: http://www.blindfiveyearold.com/crawl-optimization - he talks about using robots.txt to exclude thin content in order to increase frequency with qhich indexed content gets crawled, supposedly helping rankings. In this great whiteboard Friday, Rand suggests using "noindex, follow" - http://moz.com/blog/handling-duplicate-content-across-large-numbers-of-urls.

I am trying to get more light on this (people who have experience with this), but struggle to get answers.

Does anyone have any testing evidence what is better to use for pages with thin content, yet important pages to keep on a website? I am referring to content shared across multiple websites (such as e-commerce, real estate etc). Imagine a website with 300 high quality pages indexed and 5,000 thin product type pages, which are pages that would not generate relevant search traffic. Question goes: Does the interlinking value achieved by "noindex, follow" outweigh the negative of Google having to crawl all those "noindex" pages? With robots.txt one has Google's crawling focus on just the important pages that are indexed and that may give ranking a boost. Any experiments with insight to this would be great.

I do get the story about "make the pages unique", "get customer reviews and comments" etc....but the above question is the important question here.

Hi Pete, I just wanted to confirm, based on what you wrote:
"I don't think the picture- and video-heavy pages are going to rank all that well by themselves. It's just a question of whether those additional pages are diluting your MLS listing pages (by using similar regional keywords, etc.)."

I did following:

1. Deleted words "Home" and "Condo" from the title tag and H1 so the neighborhood name is still in title tag and H1, but no mention of home, condo, real estate etc.
2. all written content has been moved from "guides" (where pictures and videos are) to lower part of MLS result pages and I imagine over a 1-2 month period the MLS result pages will get the SEO credit for this unique written content (despite no 301 redirect)
3. I interlink from picture / video pages to MLS result pages with "neighborhood homes for sale"

My hypothesis is that over the next few months as G gets a better idea of my website (as the site gets more popular - still only 5 months old) G will know what to rank for "neighborhood homes for sale" search terms.

Makes sense?

Thats right. Zero search value. Maybe I can simply change Title tag, H1 etc. Get rid of keyword (ex "Honolulu") a d instead call ("Gallery 1"). In this way I can keep structure without diluting ranking potential for MLS result pages?

Interesting, thx. Can I do following: Add "noindex, follow" to those guide pages? In this way they wont compete w MLS result pages, which they currently do. Issue is all that geeat unique picture and video content wont be indexed by Google.....maybe not a big issue?

thx, Pete. Guides are more for users who are curious about pictures and videos - not something I care about ranking for. Ex: http://www.honoluluhi5.com/waikiki-condos-real-estate/

MLS result pages is my life and I moved a lot of written content to MLS result pages to add unique content. Ex: http://www.honoluluhi5.com/oahu/honolulu/metro/waikiki-condos/ (you will see unique content below map and thumb MLS pictures).

I feel this layout is ideal long-term. I link from guide (as you can see above) to the corresponding MLS result page. Hope this makes sense

thx a lot. "Viewing it as manipulative" - it makes sense. I will certainly refrain from doing so.

I keep saying last question, but this should be: moving some written content from Page A to Page B (yet keeping Page A, just less content remaining on Page A) is OK and will after a while be viewing as Page B's original content and Page B will get the SEO credit. This is done without a 301 re-direct, since Page A is still a page with pictures that are original and unique and I want Google to index all those pictures. Just that a bunch of unique written content was moved from Page A to Page B. I have moved written content from about 200 different guide type pages to 200 MLS result pages, as it makes more sense to have it there. Would it be safer to include the 301 re-direct and simply lose the picture indexing to play it safe?

thx. 1 last slight different, but related question: What is your view in placing written content above other content in source code, but on webpage written content displays below other content? In my case: MLS thumb pictures and descriptions (same as other realtors' websites) show on top of page and as users scroll down they see a lot of written unique original content I have. Search engines like written content higher on page, so would it be a good idea to place written content above the MLS data in the source code, though on webpage it will still display below MLS data.

thank you very much. The idea was to move a lot of great pictures from a "gallery" to a page I want to rank for. Gallery page serves no purpose but for users to see beautiful pictures and obviously for Google to index a lot of unique pictures. I guess I will leave the gallery as is and simply from the gallery inter-link to the important page.

Implementation on your suggestion can be done (my web developers have already completed, just not implemented), however, it sounds to me, if I read between the lines correctly, that there is a risk Google may screw up on interpretation of such implementation and this could potentially even hurt my site with duplicate content issues…….

Hi Pete,
There is no mechanisim that will allow a) Lots of different pictures in a slideshow only to load when users scroll to a certain part of a part yet not slowing page speed and all pictures being indexed by Google. If you can show me 1 example on the Internet that has a solution to this, I would love to see it.

This is what is possible to create (not my website, just an example): http://diveintohtml5.info/examples/history/brandy.html - I can implement such picture slideshow - which loads when users scroll down on my page - and then notice how the URL will change for each picture (as you change picture), but rest of the content on the page will stay the same. Now, the big questions go:

1. Will the main (important) URL get the SEO credit for all these other URL's where each picture is located?
2. Since each picture is on a different URL, each URL will get SEO credit separately and main URL will gain nothing from these pictures from an SEO perspective
3. Since written content is EXACTLY the same across each of these picture URL's it will look like duplicate content and it would be good to use a canonical to make sure main URL gets all SEO credit.
4. How would you place 20 unique copyrighted pictures on a URL and make sure that URL gets the SEO credit, keeping in mind the pictures can ONLY load after users scroll to a certain point on the page, as the page will otherwise load too slowly.

Highly appreciate your thoughts on this, since experts say there is a solution, but I am yet to seeing 1 concrete piece of evidence.

CORRECTION: URL 1 and URL 2 are the opposite of what I described. In other words, I want to move pictures from 1) to 2). I already moved written content from 1) to 2).

On this URL 1) http://www.honoluluhi5.com/oahu/honolulu-city-real-estate/ - you will see written content at lower part of the page. This written content was originally on this URL 2) http://www.honoluluhi5.com/oahu/honolulu-homes/. I moved it because the URL 1) is the page I want to rank for and 2) served more as a guide. I want to move the pictures from 2) as well to 1) and then add a 301 redirect. However, this is NOT possible, because if I place pictures on 1) where users only see it after scrolling down to a certain place on the URL, Google is not able to index all those pictures. Only way to index those pictures is having them load when users land on the page, which would slow down the page and be a terrible user experience.

I am told there is a solution to get these pictures indexed, but so far no one has been able to present a concrete solution.