Googlebot on paywall made with cookies and local storage
-
My question is about paywalls made with cookies and local storage. We are changing a website with free content to a open paywall with a 5 article view weekly limit.
The paywall is made to work with cookies and local storage. The article views are stored to local storage but you have to have your cookies enabled so that you can read the free articles. If you don't have cookies enable we would pass an error page (otherwise the paywall would be easy to bypass).
Can you say how this affects SEO? We would still like that Google would index all article pages that it does now.
Would it be cloaking if we treated Googlebot differently so that when it does not have cookies enabled, it would still be able to index the page?
-
Thank you for your answer!
Yes, that is exactly the case.
We have been testing this and it seems that "Googlebot" doesn't hit the wall at all with the normal settings on. With these results it seems that we don't need to treat "Googlebot" differently because it doesn't seem to store any cookie or local storage data.
Tech savvy users can bypass the pay wall by other means as well so that's not a big concern for us.
-
To make sure that I'm getting your question correct. You want Google to crawl and index all your content but you want visitors to use an open paywall that shows 5 free articles then resorts to a paywall.
Yes, it would be treated as cloaking but you have a legitimate reason for doing so and intent matters a great deal. You could check for a search engine user-agent string such as "Googlebot" and then serve the full content. This would ensure that your content is still crawled and indexed.
The only downside is any tech savvy individual can spoof the server header by setting their user-agent to "Googlebot" and bypass your paywall.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Would you rate-control Googlebot? How much crawling is too much crawling?
One of our sites is very large - over 500M pages. Google has indexed 1/8th of the site - and they tend to crawl between 800k and 1M pages per day. A few times a year, Google will significantly increase their crawl rate - overnight hitting 2M pages per day or more. This creates big problems for us, because at 1M pages per day Google is consuming 70% of our API capacity, and the API overall is at 90% capacity. At 2M pages per day, 20% of our page requests are 500 errors. I've lobbied for an investment / overhaul of the API configuration to allow for more Google bandwidth without compromising user experience. My tech team counters that it's a wasted investment - as Google will crawl to our capacity whatever that capacity is. Questions to Enterprise SEOs: *Is there any validity to the tech team's claim? I thought Google's crawl rate was based on a combination of PageRank and the frequency of page updates. This indicates there is some upper limit - which we perhaps haven't reached - but which would stabilize once reached. *We've asked Google to rate-limit our crawl rate in the past. Is that harmful? I've always looked at a robust crawl rate as a good problem to have. Is 1.5M Googlebot API calls a day desirable, or something any reasonable Enterprise SEO would seek to throttle back? *What about setting a longer refresh rate in the sitemaps? Would that reduce the daily crawl demand? We could set increase it to a month, but at 500M pages Google could still have a ball at the 2M pages/day rate. Thanks
Intermediate & Advanced SEO | | lzhao0 -
How Google organic search results differ in Local Searches?
We all know Google displays nearby results by locating our ip address. My question is how does these results differ? For eg 1. If someone from Newyork search for "chinese Restaurant in Newyork" 2. Someone from California search for "chinese Restaurant in Newyork" 3. Someone from California changes his location to Newyork and search for "chinese Restaurant in Newyork" What are the factors the Google SERP looks into to display the result in local terms?
Intermediate & Advanced SEO | | rajeevEDU0 -
How do I Improve Google Local search position
Hi, I think its called local search position, what I'm referring to is when you do a search on a keyword and google lists not only the best matches but also usually the second match is a group of 3 businesses with telephone numbers, google reviews and at the bottom of the group it will say something like: "See results for <your keyword="">on a map. This is what I'm referring to. in anycase my question is if I click on the link to see more results on a map I'm listed as number 3, however on the search page before where the link is displayed which I just clicked on I'm not being listed and instead one business name is being listed three times, each of the listings uses the same address but a different telephone number, In addtion the business that is being listed three times is also listed in the results being returned above in this case position #1 for the keyword I have searched. I assume this has something to do with them also being listed in the group of local businesses below three time.. The business I'm interested in getting listed in this group of results is currently being listed page 2 position 5 for the keyword..</your> Any suggestions would be greatly appreciated.. Thanks in advance..
Intermediate & Advanced SEO | | robdob11 -
Site Structured Navigated by Cookies
Is it advisable to have a site structure that is navigated via URLs rather than cookies? In a website that has several location based pages - each with their own functions and information? Is this a SEO priority? Will it help to combat duplicate content? Any help would be greatly appreciated!
Intermediate & Advanced SEO | | J_Sinclair0 -
Local Showroom Tel Showing Up in SERP for Branded Term Search?
We encountered an issue: when user search for a branded term in UK on Google UK, one of the showroom got listed in SERP under site links as a location with Location icon and takes over a few lines - their address and phone number are displayed. This caused confusion for user as it's only one of our showrooms and we don't want to direct calls to the showroom this way. I'm suspecting somehow the localization factor came into play but not sure how to remove the showroom phone number from being displayed. Anyone has any insight/experience with this? Thanks
Intermediate & Advanced SEO | | b.digi0 -
How do Nation wide business win Local?
Hi, What is the methodology for a nation wide e-commerce site to win local searches? If we take for example Flower Shops...
Intermediate & Advanced SEO | | BeytzNet
How do companies like 1800Flowers, ProFlowers compete locally for searches like "Flowers New York", "Flowers Boston" etc.? Should they create a dedicated page (or pages / articles) per area ? Thanks0 -
Universal Search vs Local Organic
Hi, My web site has high rankings in universal SERP's. However, in my city organic search the competitors’ web sites that even don’t show up in universal Serp’s have higher rankings than mine. Not sure what I’m doing wrong. Thanks for any insight.
Intermediate & Advanced SEO | | Zlhe0 -
Sitelinks in 7-pack / blended / local results
I have a client who has been ranking well in the 7-pack for local searches, for 1.5+ years. I recently noticed a competitor's Google Places link has little sitelinks attached, but my client's link doesn't have them. This makes me sad. To provide a concise question: what can I do to help my client get sitelinks along with his Google Places listing in the 7-pack / blended / local results? Some example data: My client's business is called Ambiance Dental and his website is www.mycalgarydentist.com. An example search to see what I'm talking about is "calgary family dentist". The competitor that's showing sitelinks is www.aestheticdentalstudio.ca which has a title of "Dentist in Calgary | Cosmetic Treatment in Calgary". The sitelinks you'll see are "Dr. Gordon Chee", "Links", "Dr. Alexa Geminiano". Notice that my client doesn't have the same sitelinks. Some further data: If you do a a search for "calgary aesthetic dentist" you'll see the competitor's 1-box local result (is that what it's called?) with his Google Places data and sitelinks. If you search for "calgary ambiance dentist" you'll get a similar layout SERP for my client, again with no sitelinks. My client's sitelinks: If you search for "ambiance dental calgary" you'll see that Google does offer sitelinks for his site, just not in Google Places it seems. My client's website: My client's website has the navigation coded as a list (UL) without any javascript or complicated code messing things up. The competitor's navigation is built similarly, though he has about 40 more pages in his main navigation. My client's page names are concise, which I've read helps with sitelinks, the website is coded very cleanly, the URLs of his site are clear and concise without a complicated folder structure, so it seems like we're doing everything right. I appreciate any input other mozzers can provide, and discussion on the topic. I'm sure there are others who would benefit from local sitelinks as well!
Intermediate & Advanced SEO | | Kenoshi0