Indexing content behind a login
-
Hi,
I manage a website within the pharmaceutical industry where only healthcare professionals are allowed to access the content. For this reason most of the content is behind a login.
My challenge is that we have a massive amount of interesting and unique content available on the site and I want the healthcare professionals to find this via Google!
At the moment if a user tries to access this content they are prompted to register / login. My question is that if I look for the Google Bot user agent and allow this to access and index the content will this be classed as cloaking? I'm assuming that it will.
If so, how can I get around this? We have a number of open landing pages but we're limited to what indexable content we can have on these pages!
I look forward to all of your suggestions as I'm struggling for ideas now!
Thanks
Steve
-
Thanks everyone... It's not as restrictive as patient records... Basically, because of the way our health service works in the UK we are not allowed to promote material around our medicines to patients, it should be restricted only to HCP's. If we are seen to be actively promoting to patients we run the risk of a heavy fine.
For this reason we need to take steps to ensure that we only target this information towards HCP's and therefore we require them to register before being able to access the content...
My issue is that HCP's may search for a Brand that we supply but we have to be very careful what Brand information we provide outside of log-in. Therefore the content we can include on landing pages cannot really be optimised for the keywords that they are searching for! Hence why I want the content behind log-in indexed but not easily available without registering...
It's a very difficult place to be!
-
I guess I was just hoping for that magic answer that doesn't exist! It's VERY challenging to optimise a site with these kinds of restrictions but I get I just need to put what I can on the landing pages and optimise as best I can with the content I can show!
We also have other websites aimed at patients where all the content is open so I guess I'll just have to enjoy optimising these instead
Thanks for all your input!
Steve
-
Steve,
Yes that would be cloaking. I wouldn't do that.
As Pete mentioned below, your only real options at this point are to make some of the content, or new content, available for public use. If you can't publish abstracts at least, then you'll have to invest in copywriting content that is legally available for the public to get traffic that way, and do your best to convert them into subscribers.
-
Hi Steve
If it can only be viewed legally by health practitioners who are members of your site, then it seems to me you don't have an option as by putting any of this content into the public domain on Google by whatever method you use will be deemed illegal by whichever body oversees it.
Presumably you cannot also publish short 25o word summaries of the content?
If not, then I think you need to create pages that are directly targeted at marketing the site to health practitioners. Whilst the pages won't be able to contain the content you want to have Google index, they could still contain general information and the benefits of becoming a subscriber.
Isn't that the goal of the site anyway, i.e. to be a resource to health practitioners? So, without being able to make the content public, you have to market to them through your SEO or use some other form or indirect or direct marketing to encourage them to the site to sign up.
I hope that helps,
Peter -
Thanks all... Unfortunately it is a legal requirement that the content is not made publicly available but the challenge then is how do people find it online!
I've looked at first click free and pretty much ever other option I could think of and yet to find a solution
My only option is to allow Google Bot through the authentication which will allow it to index the content but my concern is that this is almost certainly cloaking...
-
Please try looking at "First Click Free" by Google
https://support.google.com/webmasters/answer/74536?hl=en
I think this is along the lines of what you are looking for.
-
Hi Steve
As you already know, if a page is not crawlable it's not indexable. I don't think there is any way around this without changing the strategy of the site. You said, _"We have a number of open landing pages but we're limited to what indexable content we can have on these pages". _Is that limitation imposed by a legal requirement or something like that, or by the site owners because they don't want to give free access?
If the marketing strategy for the site is to grow the membership, then as it's providing a content service to its members then it has to give potential customers a sample of its wares.
I think there are two possible solutions.
(1) increase the amount of free content available on the site to give the search engines more content to crawl and make available to people searching or
(2) Provide a decent size excerpt, say the first 250 words of each article as a taster for potential customers and put the site login at the point of the "read more". That way you give the search engines something to get their teeth into which is of a decent length but it's also a decent size teaser to give potential customers an appetite to subscribe.
I hope that helps,
Peter
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Duplicate Content for e-commerce help
Hi. I know I have duplicate content issues and Moz has shown me the issues on ecommerce websites. However a large number of these issues are for variations of the same product. For example a blue, armani t-shirt can be found on armani page, t-shirt page, armani t-shirt page and it also shows links for the duplicates due to sizing variations. Is it possible or even worthwhile working on these issues? Thanks
White Hat / Black Hat SEO | | YNWA0 -
Cloaking for better user experience and deeper indexing - grey or black?
I'm working on a directory that has around 800 results (image rich results) in the top level view. This will likely grow over time so needs support thousands. The main issue is that it is built in ajax so paginated pages are dynamically generated and look like duplicate content to search engines. If we limit the results, then not all of the individual directory listing pages can be found. I have an idea that serves users and search engines what they want but uses cloaking. Is it grey or black? I've read http://moz.com/blog/white-hat-cloaking-it-exists-its-permitted-its-useful and none of the examples quite apply. To allow users to browse through the results (without having a single page that has a slow load time) we include pagination links but which are not shown to search engines. This is a positive user experience. For search engines we display all results (since there is no limit the number of links so long as they are not spammy) on a single page. This requires cloaking, but is ultimately serving the same content in slightly different ways. 1. Where on the scale of white to black is this? 2. Would you do this for a client's site? 3. Would you do it for your own site?
White Hat / Black Hat SEO | | ServiceCrowd_AU0 -
Lots of websites copied my original content from my own website, what should I do?
1. Should I ask them to remove and replace the content with their unique and original content? 2. Should I ask them to link to the URL where the original content is located? 3. Should I use a tool to easily track these "copycat" sites and automatically add links from their site to my site? Thanks in advance!
White Hat / Black Hat SEO | | esiow20130 -
Is it still valuable to place content in subdirectories to represent hierarchy or is it better to have every URL off the root?
Is it still valuable to place content in subdirectories to represent hierarchy on the site or is it better to have every URL off the root? I have seen websites structured both ways. It seems having everything off the root would dilute the value associated with pages closest to the homepage. Also, from a user perspective, I see the value in a visual hierarchy in the URL.
White Hat / Black Hat SEO | | belcaro19860 -
How does Google decide what content is "similar" or "duplicate"?
Hello all, I have a massive duplicate content issue at the moment with a load of old employer detail pages on my site. We have 18,000 pages that look like this: http://www.eteach.com/Employer.aspx?EmpNo=26626 http://www.eteach.com/Employer.aspx?EmpNo=36986 and Google is classing all of these pages as similar content which may result in a bunch of these pages being de-indexed. Now although they all look rubbish, some of them are ranking on search engines, and looking at the traffic on a couple of these, it's clear that people who find these pages are wanting to find out more information on the school (because everyone seems to click on the local information tab on the page). So I don't want to just get rid of all these pages, I want to add content to them. But my question is... If I were to make up say 5 templates of generic content with different fields being replaced with the schools name, location, headteachers name so that they vary with other pages, will this be enough for Google to realise that they are not similar pages and will no longer class them as duplicate pages? e.g. [School name] is a busy and dynamic school led by [headteachers name] who achieve excellence every year from ofsted. Located in [location], [school name] offers a wide range of experiences both in the classroom and through extra-curricular activities, we encourage all of our pupils to “Aim Higher". We value all our teachers and support staff and work hard to keep [school name]'s reputation to the highest standards. Something like that... Anyone know if Google would slap me if I did that across 18,000 pages (with 4 other templates to choose from)?
White Hat / Black Hat SEO | | Eteach_Marketing0 -
Ways to find private - non-indexed forums in a niche
I would wondering if there were ways to find non-indexed content in private forums/discussion boards. Is there a scalable 'foot-print' that suggests the forum has a private section?
White Hat / Black Hat SEO | | ilyaelbert0 -
Methods for getting links to my site indexed?
What are the best practices for getting links to my site indexed in search engines. We have been creating content and acquiring backlinks for the last few months. They are not being found in the back link checkers or in the Open Site Explorer. What are the tricks of the trade for imporiving the time and indexing of these links? I have read about some RSS methods using wordpress sites but that seems a little shady and i am sure google is looking for that now. Look forward to your advice.
White Hat / Black Hat SEO | | devonkrusich0 -
My attempt to reduce duplicate content got me slapped with a doorway page penalty. Halp!
On Friday, 4/29, we noticed that we suddenly lost all rankings for all of our keywords, including searches like "bbq guys". This indicated to us that we are being penalized for something. We immediately went through the list of things that changed, and the most obvious is that we were migrating domains. On Thursday, we turned off one of our older sites, http://www.thegrillstoreandmore.com/, and 301 redirected each page on it to the same page on bbqguys.com. Our intent was to eliminate duplicate content issues. When we realized that something bad was happening, we immediately turned off the redirects and put thegrillstoreandmore.com back online. This did not unpenalize bbqguys. We've been looking for things for two days, and have not been able to find what we did wrong, at least not until tonight. I just logged back in to webmaster tools to do some more digging, and I saw that I had a new message. "Google Webmaster Tools notice of detected doorway pages on http://www.bbqguys.com/" It is my understanding that doorway pages are pages jammed with keywords and links and devoid of any real content. We don't do those pages. The message does link me to Google's definition of doorway pages, but it does not give me a list of pages on my site that it does not like. If I could even see one or two pages, I could probably figure out what I am doing wrong. I find this most shocking since we go out of our way to try not to do anything spammy or sneaky. Since we try hard not to do anything that is even grey hat, I have no idea what could possibly have triggered this message and the penalty. Does anyone know how to go about figuring out what pages specifically are causing the problem so I can change them or take them down? We are slowly canonical-izing urls and changing the way different parts of the sites build links to make them all the same, and I am aware that these things need work. We were in the process of discontinuing some sites and 301 redirecting pages to a more centralized location to try to stop duplicate content. The day after we instituted the 301 redirects, the site we were redirecting all of the traffic to (the main site) got blacklisted. Because of this, we immediately took down the 301 redirects. Since the webmaster tools notifications are different (ie: too many urls is a notice level message and doorway pages is a separate alert level message), and the too many urls has been triggering for a while now, I am guessing that the doorway pages problem has nothing to do with url structure. According to the help files, doorway pages is a content problem with a specific page. The architecture suggestions are helpful and they reassure us they we should be working on them, but they don't help me solve my immediate problem. I would really be thankful for any help we could get identifying the pages that Google thinks are "doorway pages", since this is what I am getting immediately and severely penalized for. I want to stop doing whatever it is I am doing wrong, I just don't know what it is! Thanks for any help identifying the problem! It feels like we got penalized for trying to do what we think Google wants. If we could figure out what a "doorway page" is, and how our 301 redirects triggered Googlebot into saying we have them, we could more appropriately reduce duplicate content. As it stands now, we are not sure what we did wrong. We know we have duplicate content issues, but we also thought we were following webmaster guidelines on how to reduce the problem and we got nailed almost immediately when we instituted the 301 redirects.
White Hat / Black Hat SEO | | CoreyTisdale0