Thankfully Webmaster World were able to provide some decent information, for those of you who have arrived here looking for a similar answer.

There is something called the "hash-bang" which makes javascript pages crawlable.  Hashbang refers to hash (#) bang (!) - so an example would be example.com/#!/page-1.

Here's a great place to read more, understand and learn to implement:

http://support.google.com/webmasters/bin/answer.py?hl=en&answer=174992

Cheers all!

Here's an example of a # URL which has not  been indexed.

http://dulas.org.uk/hydro-info.cfm#specification_installation

Unlike the site I am working on, this site 'hides' content from the user until they click on a particular tab.  All of the original code is in the source for http://dulas.org.uk/hydro-info.cfm but only shown to the user if they activate the particular piece of javascript when they click on a tab.

The site I am working on is different - it loads content based on javascript, however it essentially loads as a new page - the content is not present in the source until you click no something, when new content will load and the old content will disappear.

Perhaps Google will be able to see that these # pages function much like a normal page, loading completely new content and getting rid of old content, and may therefore index them if I submit them in a sitemap.  However, I'd like to hear from somebody who can tell me they have done this and had success!

Thanks,

Mark

Hi Lee,

Thanks for your response. My concern is that # URLs tend to send users to a particular location on a page, rather than a new page itself.  Therefore, some things I have read suggest that Google has adapted to ignore anything after a # in order to avoid indexing an enormous amount of duplicate content.  Strange that there is so much conflicting info out there!

Cheers,

Mark

Hi there,

I have a potential issue with a site whereby all pages are dynamically populated using Javascript.  Thus, an example of an URL on their site would be www.example.com/#!/category/product.

I have read lots of conflicting information on the web - some says Google will ignore everything after the hashtag; other people say that Google will now index everything after the hashtag.

Does anybody have any conclusive information about this?  Any links to Google or Matt Cutts as confirmation would be brilliant.

P.S. I am aware about the potential issue of duplicate content, but I can assure you that has been dealt with.  I am only concerned about whether Google will index full URLs that contain hashtags.

Thanks all!

Mark

AH!  You did it Keith - I thought clicking 'update' at the bottom would do it, but there's a little link hidden in some text at the top saying "rebuild the sitemap manually".

Finally it's done, thanks so much for your help!

Mark

I'm just trying to exclude these child URLs from the sitemap - in future I may block them entirely, but I certainly don't want to submit a sitemap with these URLs and then contradict that in robots.txt.

I have used the Post ID numbers to exclude the pages from the sitemap, however they remain in place.

Thanks once again for your assistance and quick responses!

Hi Keith,

Thanks once again for a quick response.  I have actually tried that method, however when I check the live sitemap I can still see the pages in my sitemap.  Very frustrating!  Is it that the sitemap doesn't update live straight away?  And just to confirm, I am clicking "Update Options" at the bottom - quite often it'll be something stupid like that!

Thanks,

Mark

Hi all,

I was recommended the XML Sitemap Generator for Wordpress by the very helpful Keith Bloemendaal and John Pring - however I can't seem to exclude child URLs.

There is a section Exclude items  and a subsection Exclude posts.  I have tried inputting the URLs for the pages I don't want in the sitemap, however that didn't work.  So I read that you have to include a list of "IDs" - not sure where on earth to find that info, tried the page name and the post= number from the URL, however neither worked.

I hope somebody can point me in the right direction - and apologies, I am a Wordpress novice, and I got no answers from the Wordpress forums so turned right back to SEOmoz!

Cheers.

AH!  You did it Keith - I thought clicking 'update' at the bottom would do it, but there's a little link hidden in some text at the top saying "rebuild the sitemap manually".

Finally it's done, thanks so much for your help!

Mark

Thankfully Webmaster World were able to provide some decent information, for those of you who have arrived here looking for a similar answer.

There is something called the "hash-bang" which makes javascript pages crawlable.  Hashbang refers to hash (#) bang (!) - so an example would be example.com/#!/page-1.

Here's a great place to read more, understand and learn to implement:

http://support.google.com/webmasters/bin/answer.py?hl=en&answer=174992

Cheers all!