Can Google read content/see links on subscription sites?
-
If an article is published on The Times (for example), can Google by-pass the subscription sign-in to read the content and index the links in the article?
Example: http://www.thetimes.co.uk/tto/life/property/overseas/article4245346.ece
In the above article there is a link to the resort's website but you can't see this unless you subscribe. I checked the source code of the page with the subscription prompt present and the link isn't there.
Is there a way that these sites deal with search engines differently to other user agents to allow the content to be crawled and indexed?
-
Hey Matt,
The best way to tell what the news organization or site is using is to turn off javascript or view the google cache to determine how Google "sees" the page.
This article is using the second option in the article I mentioned - snippets. Here is what the article has to say about that:
"If you prefer this option, please display a snippet of your article that is at least 80 words long and includes either an excerpt or a summary of the specific article." -
Thanks Dan, it doesn't look like the example article is using first click free. So I guess the answer is no, Google can't read the hidden content in this example?
-
Great question! Yes, Google has an effective way to deal with this since 2007. The three ways they deal with this include first click free, subscription designation, and then disallowing content. Here is their official support article on it:
https://support.google.com/news/publisher/answer/40543?hl=en
Here is a quote from the help article:
"To summarize, we will crawl and index your site to the extent that you allow Googlebot to access it. In order to provide the best possible user experience and help more users discover your content, we encourage you to try First Click Free. If you prefer to limit access to your site to subscribers only, we will respect your decision and show a “subscription” label next to your links on Google News."Here is what Matt Cutts said about it in an interview with Search Engine Land:
"First Click Free originated with Google News, but you can use the same way of handling content in web search (show the same page to users and Googlebot, then if the user clicks to read a different article, then you can show them the registration or pay page). Because the same page is presented to users and to Googlebot, it’s not cloaking. So First Click Free is a great way if you have premium content to surface it in Google’s web index without cloaking. Hope that makes sense."It is possible to allow the Googlebot to access the content and simultaneously NOT provide it for free to non-subscribers. The above help article above should answer all of your questions. Hope this helps!
-
I would say no. The content of the article other than what is seen is not in the source code. They could be showing something different to Google, but if they did it would be against Google's terms of service. https://support.google.com/webmasters/answer/66355?hl=en
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Indexed Site A's Content On Site B, Site C etc
Hi All, I have an issue where the content (pages and images) of Site A (www.ericreynolds.photography) are showing up in Google under different domains Site B (www.fastphonerepair.com), Site C (www.quarryhillvet.com), Site D (www.spacasey.com). I believe this happened because I installed an SSL cert on Site A but didn't have the default SSL domain set on the server. You were able to access Site B and any page from Site A and it would pull up properly. I have since fixed that SSL issue and am now doing a 301 redirect from Sites B, C and D to Site A for anything https since Sites B, C, D are not using an SSL cert. My question is, how can I trigger google to re-index all of the sites to remove the wrong listings in the index. I have a screen shot attached so you can see the issue clearer. I have resubmitted my site map but I'm not seeing much of a change in the index for my site. Any help on what I could do would be great. Thanks
Intermediate & Advanced SEO | | cwscontent
Eric TeVM49b.png qPtXvME.png1 -
Google Is Indexing my 301 Redirects to Other sites
Long story but now i have a few links from my site 301 redirecting to youtube videos or eCommerce stores. They carry a considerable amount of traffic that i benefit from so i can't take them down, and that traffic is people from other websites, so basically i have backlinks from places that i don't own, to my redirect urls (Ex. http://example.com/redirect) My problem is that google is indexing them and doesn't let them go, i have tried blocking that url from robots.txt but google is still indexing it uncrawled, i have also tried allowing google to crawl it and adding noindex from robots.txt, i have tried removing it from GWT but it pops back again after a few days. Any ideas? Thanks!
Intermediate & Advanced SEO | | cuarto7150 -
If linking to contextual sites is beneficial for SE rankings, what impact does the re=“nofollow” attribute have when applied to these outbound contextual links?
Communities, opinion-formers, even Google representatives, seem to offer a consensus that linking to quality, relevant sites is good practice and therefore beneficial for SEO. Does this still apply when the outbound links are "nofollow"? Is there any good research on this out there?
Intermediate & Advanced SEO | | danielpressley0 -
Unnatural links to your site—impacts links
I got message in my Google webmaster tool: Unnatural links to your site—impacts links Does anyone knows the difference between "Unnatural links to your site—impacts links" and "Unnatural links to your site" Thank you Sina
Intermediate & Advanced SEO | | SinaKashani0 -
Main content - javascript/ajax
Hi, On most of our pages Javascript is displaying our main content, so it doesn't show up on the page source and I assume isn't being crawled by Google to the best of its ability. It's also not showing up on MOZ's page grader and crawl results, making analysis and testing harder. What's the easiest way, without having to completely redo our website, to have this content crawled by search engines and moz?
Intermediate & Advanced SEO | | S.S.N0 -
Site wide internal links in footer
I have had a long discussion with a client and their external SEO partner about their current footer. They have added all their product categories, both main and sub, to the footer. From a pure SEO perspective is it still advisable, after all the pandas and penguines, to stay away from keyword important site wide footer linking to internal pages? As the links will become a repeatable element and also containing the most important keywords, isn't the links actually hurting more than helping? With 5000 index pages, it will risk "marking" the most important keywords as repeatable, lowering ranking, instead of increasing as their external part say.
Intermediate & Advanced SEO | | Macaper1 -
Outbound Links to Authority sites
Will outbound links to a related topic on an authority site help, hurt or be irrelevanent for SEO purposes. And if beneficially, should it be Nofollow?
Intermediate & Advanced SEO | | VictorVC0 -
First Link Priority question - image/logo in header links to homepage
I have not found a clear answer to this particular aspect of the "first link priority" discussion, so wanted to ask here. Noble Samurai (makers of Market Samurai seo software) just posted a video discussing this topic and referencing specifically a use case example where when you disable all the css and view the page the way google sees it, many times companies use an image/logo in their header which links to their homepage. In my case, if you visit our site you can see the logo linking back to the homepage, which is present on every page within the site. When you disable the styling and view the site in a linear path, the logo is the first link. I'd love for our first link to our homepage include a primary keyword phrase anchor text. Noble Samurai (presumably seo experts) posted a video explaining this specifically http://www.noblesamurai.com/blog/market-samurai/website-optimization-first-link-priority-2306 and their suggested code implementations to "fix" it http://www.noblesamurai.com/first-link-priority-templates which use CSS and/or javascript to alter the way it is presented to the spiders. My web developer referred me to google's webmaster central: http://www.google.com/support/webmasters/bin/answer.py?answer=66353 where they seem to indicate that this would be attempting to hide text / links. Is this a good or bad thing to do?
Intermediate & Advanced SEO | | dcutt0