Free Article Previews (Free Registration Wall)
-
My site is www.optionmonster.com It has a few different areas of "Free" and "Premium" content.
1. We have a first click free program for Google visitors, where they can read the first news article in full.
2. Other visitors and subsequent news article views have a content preview, which includes headline and first paragraph of content, and then asks for a name and email address in order to consume the rest of the free news.
3. We have premium content (paid), and all of those folders are currently on our robots.txt file
My questions are as follows....
a. How much content is needed in front of the "pay/reg wall" in order to still get proper juice from linkbacks/pagerank crawls
b. Should I make the premium content a robots meta noindex tag, or because you can't see the article at all without logging in, is a proper place to include on robots.txt
-
Since GoogleBots can't access the premium regardless, is it redundant to have on the robots.txt file?
You are indeed correct the best practice is to minimize your robots.txt file. That is why the very first sentence in my initial response was the suggestion to NOT include your premium sections in the robots.txt file
Will google and other bots will hurt you if you have premium sections that they try to crawl but can't?
No.
-
Okay. But since GoogleBots can't access the premium regardless, is it redundant to have on the robots.txt file? I thought the best practice was to have almost nothing on the robots.txt file per (http://www.seomoz.org/blog/serious-robotstxt-misuse-high-impact-solutions)
So my question comes down to if google and other bots will hurt you if you have premium sections that they try to crawl but can't, vs ignoring them because they are on the robots.txt
-
So you wouldn't No index FOLLOW those pages, in case someone links to them?
I am not sure what I said to offer that impression.
By "those pages" I am guessing you are referring to your premium content? If so, it really doesn't matter what you do with the content as Google can't get to it due to the robots.txt file. Even if you change the meta tag, Google wont be able to see the change.
Yes, it is possible they could try to follow a link but since this is premium content, they wont be able to access it.
Perhaps I am not clear on your Free vs Premium division. In short, any content which you do not desire to be indexed should contain the NoIndex tag, but that only applies if a crawler has access to the content.
-
So you wouldn't No index FOLLOW those pages, in case someone links to them?
-
I would not include your premium area in the robots.txt file at all. Google will recognize that they are unable to access the content, and it won't get indexed. There is no need to block it further.
For your other content, as long as you are showing Google the same exact information as you are showing to other visitors, you are fine. If you show Google more then you do to other visitors, then your site can be blacklisted.
How much content is needed in front of the "pay/reg wall" in order to still get proper juice from linkbacks/pagerank crawls
There are many factors involved. There isn't an answer like "400 words and you are good". These type of setups can be tricky. If you have a forum, then you can allow visitors to see the first thread but not any of the replies. By limiting content based on a number of words, you will be cutting off some articles, and it is possible a small article can be viewed in it's entirety.
It is more important your key words are in the content. If I linked to an article with the anchor text of 1980 Corvette, and your article started off discussing cars but you cut it off before discussing the 1980 Corvette, then that link can be devalued.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google rejected my reconsideration request of unnatural link manual action, and list one blog article twice as example?
Hi Moz Community, On April 22 my site received a manual action in Google Webmaster telling me it's caused by unnatural links. After some a deep cleaning of all the sitewide links, which I think is the major problem of my external links, I requested a reconsideration request on May 4. And Google rejected my reconsideration request of unnatural link manual action on May 29, and list one blog article twice as example, which is quite weird to me. Is it normal for Google to list one URL twice as example in the feedback? I don't quite see the reason for that. Does anybody have any idea about that? This is really quite frustrating to me. And to be honest, I don't see much problems about the article Google listed as well. Yeah it's all about our product and it has 3 do-follow links to our site. But it contains no words such as sponsor, advertisement, or rewards... And the blog itself is quite healthy as well. The post also get rather high engagement, with organic comments and shares. How did Google flag that out? I don't think it's possible that Google will go into all our site links one by one... Hope you guys can help me with that. Thanks in advance! Ben
Technical SEO | | Ben_fotor0 -
Unpublish and republish old articles
This might be a dumb question but we had an incident where a new SEO guy thought it would be a good idea to un-publish and republish all of your 200+ blog posts which we carefully scheduled over the last 6 months. He did not update the content and did not change anything. His intention was to send out google a sign to recheck the sites or something. Now, the entire blog looks like it wen't live in one day, which I don't think is good? Should we load a backup and get our old publishing dates back, should we keep it with the new publishing dates? What are the consequences? Will it effect our SEO?
Technical SEO | | Funlocity1 -
JSON-LD meta data: Do you have any rules/recommendations for using BlogPosting vs Article?
Dear Moz Community. I'm looking at moving from in-line Microdata in the HTML to JSON-LD on the web pages that I manage. Seems a far simpler solution having all the meta data in one place - especially for trouble shooting! With this in mind I've started to change the page templates on my personal site before I tackle the ones for my eCommerce site. I've made a start, and I'm still working on the templates producing some default values (like if a page doesn't have an associated image) but have been wondering if any of you have any rules/recommendations for using BlogPosting vs Article? I'd call this type of page an Article:
Technical SEO | | andystorey
https://cycling-jersey-collection.com/browse-collection/selle-italia-chinol-seb-bennotto-1982-team-jersey Whereas this page is from the /blog so that should probably be a BlogPosting:
https://cycling-jersey-collection.com/blog/2017-worldtour-team-jerseys I've used the following resources but it would be great to get a discussion on here.
https://yoast.com/structured-data-schema-ultimate-guide/
https://developers.google.com/search/docs/data-types/data-type-selector
https://search.google.com/structured-data/testing-tool/u/0/ I'm keen to get this 100% right as once this is done I'm going to drive through some further changes to get some progress on things like this: https://moz.com/blog/ranking-zero-seo-for-answers
https://moz.com/blog/what-we-learned-analyzing-featured-snippets Kind Regards andy moz-screenshot.jpg1 -
For a web agency, can the same WHOIS registrant name information between different domain names hurt SEO?
Hi, I own a web agency and we often register domain names for our clients. When we register those, I just realized that we always use the same registrant name and other whois (ex : Louis-Philippe Dea), for all domain names. Can this hurt the SEO of our clients? Hope that you'll understand when I'm trying to say in spite of my bad english :S Thanks 🙂
Technical SEO | | Louis-Philippe_Dea0 -
How is Google finding our preview subdomains?
I've noticed that Google is able to find, crawl and index preview subdomains we set up for new client sites (e.g. clientpreview.example.com). I know now to use "meta name="robots" and robots.txt) to block the search engines from crawling these subdomains. My question though, is how is Google finding these subdomains? We don't link to these preview domains from anywhere else, so I can't figure out how Google is even getting there. Does anybody have any insight on this?
Technical SEO | | ZeeCreative0 -
Young site trying hard, but banging head against the wall -- Site Review
Hi All New to PRO but we're seriously committed to getting this working. And firstly thank you to anyone who offers any useful thoughts and insights. We've launched a new site, unfortunately late to the market for the season and are really struggling to get search engine recognition. Site: http://www.ignitehats.co.uk/ We're continuously adding new content, slowly gathering more links and working hard to promote socially. But even on our clearest search terms like "Ignite hats" we're down on page 4. Both GWT and the Seomoz tools highlight no big problems (a few titles that are too long) but otherwise nothing. Maybe wrongly we requested that the Google spam team review our site incase it was being penalised, but got a template response saying the site was not in their spam system (phew, there wasn't a reason it should be we believe). We're wondering if this is just that our site is just too young? It's been live for 6 weeks. But worry maybe this is not the case. We've had success with another site we run much sooner than this. Any help or pointers would be really appreciated. Similar stories and what others have done, at least to give us some confidence to carry on would be great. Thanks for reading.
Technical SEO | | JHill0 -
Have I Set my joomla site map correctly as google not picking up articles
Hi here is my site map http://www.in2town.co.uk/sitemap-xml?sitemap=1 But i am concerned that i have not set my joomla site map correctly. The reason why i think this is, because i have a lot of articles on the site and google are not picking them up in my google webmaster tools. On my old site google was quick to pick up all the articles. Can anyone offer me any help on this and let me know if my site map should be set to show all of my articles
Technical SEO | | ClaireH-1848860 -
How do i can change my instant preview image?
Since we've put an index.htm file indicating that our webiste is under reconstruction and have removed it few hours after, the instant preview picture still showing the informative message. It has been 40 day since and we've submit our sitemap few times. Is there a way to change it manually?
Technical SEO | | webit400