Client Portal and SEO Considerations?

[Brian_Dowd]

Hi Moz and Moz fans,

We are looking to add a client portal to the website. Basically, I haven't found too much on this with regard to SEO. The idea would be that certain parts of the website would be hidden under a pay wall and for subscribers, they would be able to see all content. I am wondering if anyone has any experience with that and what SEO considerations to take into account.

One thing we are particularly concerned about is how Google will index the portions of the website behind the pay wall, if at all. Obviously, we would rather that they don't index it, so that people can't find a way to get to the info without paying. I would imagine it would have to do with the type of coding, however, I am not a coding guru, so I am not 100% on that.

Anyway, anyone that has any experience in this kind of thing and can comment on this at all, any comment is welcome. Also, any documentation that could be helpful would be welcome too.

Thanks

[Brian_Dowd]

Hi Sergey,

Thanks for this. I have read that article at some point a while back. Interesting read. I was just sort of sounding out a few ideas in my last post. The second option seems to be the best way to go.

Do you know of any other methods to do this that is common in the industry?

Thanks,
Brian

[sergeystefoglo]

Hi Brian,

Google can definitely crawl Javascript, although it is limited in a sense. Here is a great article addressing this.

I have seen many websites use the solution that you mention in your second paragraph. That is, creating a separate section of the domain (or creating a subdomain) for users who are logging in. GoogleBot will not be able to crawl anything that is beyond a login portal, so this is the most common solution. Of course you can also disallow the specific path or subdomain in robots.txt as well!

It comes down to how much content that you have behind the login portal. You want to figure out which part of your site you want to be crawled, and which parts you don't want. After that point it's a matter of figuring out the best solution for blocking that content from crawlers.

Hope that helps!

[Brian_Dowd]

Hi Andy,

Thanks for the reply.

This documentation was helpful. You mentioned the potential for the content to be seen in the cached version of pages. I read the Search Engine Land article there and shes sort of getting at how you don't want to block your content to search engines because then they can't be indexed and ranked. Let's say we don't really care about those parts of the pages that we wan't to block in terms of being indexed, which I sort of mentioned above. If we were to some how display those parts of the pages, images and text, in JavaScript, search engines wouldn't be able to crawl. Would those portions of the page still be visible in the cached version? I would imagine not if the search engines can't crawl JS, but not sure.

A second option I thought of was to create a separate subdomain for logged in users that would only be accessible through a log in form. Search engines cannot follow forms, so I would imagine, that gated content wouldn't be visible to crawlers and therefore not indexed. You could go one step farther and block the subdomain in robots.txt. Do you think that could be a possible solution?

Thanks again for the reply last week.

-Brian

[Andy.Drinkwater]

Hi Brian,

I have a client working on correcting this issue with his site at the moment. They run a big media site that allows access once paid, but so may of these sites suffer with the same issue and because they allow Google to index the whole text, but only show a portion of it, this means that if you just look at the cached version, you can read it without paying.

In terms of correcting it, I would first have a read on how Google handles subscription sites. You can find that info here. Google prefers the "First click free" model.

There is additional reading on this subject over at Search Engine Land. First Click Free (FCF) is what you want to be looking into in more detail.

I hope this helps a little.

-Andy