Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
Will a disclaimer affect Crawling?
-
Hello everyone!
My German users will have to get a disclaimer according to German laws, now my question is the following:
Will a disclaimer affect crawling? What's the best practice to have regarding this? Should I have special care in this? What's the best disclaimer technique? A Plain HTML page? Something overlapping the site?
Thank you all!
-
Hi friend, you can display the disclaimer using a JavaScript overlay and this would be absolutely fine. The bots won't have any trouble crawling the website behind the JS overlay as they won't see it. This is a very common practice among the websites that display age gate verification page like porn sites and sites that talk or sell liquor etc..
This technique is not considered cloaking as the intention is not malicious or deceptive and Google handles these normally. Hope it helps and Good Luck.
I addressed a similar question here on Moz:
http://moz.com/community/q/different-user-experience-with-javascript-on-off
Best regards,
Devanur Rafi
-
Maybe I will try as you said, will just wait to see if someone else responds so I can gather more ideas. Thanks though!
About cookies, yes, it's an Europe thing, but in Germany if you have an adult site, if you sell some type of products, etc, you have to display a disclaimer
-
Hmm, I honestly do not know in this situation. One thing you might try is to do a modal that blocks the page with a semi transparent layer, but check if it is googlebot accessing the site and not do a modal.
But honestly, I thought it was a cookies thing being in the EU so I am not an expert in this area.
-
Thanks for the input!
while the site will not be pornographic it will include art nudity and I want to have a disclaimer that covers at least a portion of teh page -
Don't block the site totally and it will not matter really. A lot of people in the e-commerce world do it like in this demo, http://warehouse.iqit-commerce.com/selector/?theme=warehouse2 Just a small bar on the bottom of the page. If you wanted to get even more clever, you could geographically target the user and show based on that and exclude bots from seeing it. But I would not suggest blocking the whole page like an adult site does if it is for cookies. If it is an adult site, that needs a full disabling disclaimer, I have no experience in that area.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Does having alot of pages with noindex and nofollow tags affect rankings?
We are an e-commerce marketplace at for alternative fashion and home decor. We have over 1000+ stores on the marketplace. Early this year, we switched the website from HTTP to HTTPS in March 2018 and also added noindex and nofollow tags to the store about page and store policies (mostly boilerplate content) Our traffic dropped by 45% and we have since not recovered. We have done I am wondering could these tags be affecting our rankings?
Intermediate & Advanced SEO | | JimJ1 -
How will changing my website's page content affect SEO?
Our company is looking to update the content on our existing web pages and I am curious what the best way to roll out these changes are in order to maintain good SEO rankings for certain pages. The infrastructure of the site will not be modified except for maybe adding a couple new pages, but existing domains will stay the same. If the domains are staying the same does it really matter if I just updated 1 page every week or so, versus updating them all at once? Just looking for some insight into how freshening up the content on the back end pages could potentially hurt SEO rankings initially. Thanks!
Intermediate & Advanced SEO | | Bankable1 -
[Very Urgent] More 100 "/search/adult-site-keywords" Crawl errors under Search Console
I just opened my G Search Console and was shocked to see more than 150 Not Found errors under Crawl errors. Mine is a Wordpress site (it's consistently updated too): Here's how they show up: Example 1: URL: www.example.com/search/adult-site-keyword/page2.html/feed/rss2 Linked From: http://an-adult-image-hosting.com/search/adult-site-keyword/page2.html Example 2 (this surprised me the most when I looked at the linked from data): URL: www.example.com/search/adult-site-keyword-2.html/page/3/ Linked From: www.example.com/search/adult-site-keyword-2.html/page/2/ (this is showing as if it's from our own site) http://a-spammy-adult-site.com/search/adult-site-keyword-2.html Example 3: URL: www.example.com/search/adult-site-keyword-3.html Linked From: http://an-adult-image-hosting.com/search/adult-site-keyword-3.html How do I address this issue?
Intermediate & Advanced SEO | | rmehta10 -
Will I lose Link Juice when implementing a Reverse Proxy?
My company is looking at consolidating 5 websites that it has running on magento, wordpress, drupal and a few other platforms on to the same domain. Currently they're all on subdomains but we'd like to consolidate the subdomains to folders for UX and SEO potential. Currently they look like this: shop.example.com blog.example.com uk.example.com us.example.com After the reverse proxy they'll look like this: example.com/uk/ example.com/us/ example.com/us/shop example.com/us/blog I'm curious to know how much link juice will be lost in this switch. I've read a lot about site migration (especially the Moz example). A lot of these guides/case studies just mention using a bunch of 301's but it seems they'd probably be using reveres proxies as well. My questions are: Is a reverse proxy equal to or worse/better than a 301? Should I combine reverse proxy with a 301 or rel canonical tag? When implementing a reverse proxy will I lose link juice = ranking? Thanks so much! Jacob
Intermediate & Advanced SEO | | jacob.young.cricut0 -
Mobile Googlebot vs Desktop Googlebot - GWT reports - Crawl errors
Hi Everyone, I have a very specific SEO question. I am doing a site audit and one of the crawl reports is showing tons of 404's for the "smartphone" bot and with very recent crawl dates. If our website is responsive, and we do not have a mobile version of the website I do not understand why the desktop report version has tons of 404's and yet the smartphone does not. I think I am not understanding something conceptually. I think it has something to do with this little message in the Mobile crawl report. "Errors that occurred only when your site was crawled by Googlebot (errors didn't appear for desktop)." If I understand correctly, the "smartphone" report will only show URL's that are not on the desktop report. Is this correct?
Intermediate & Advanced SEO | | Carla_Dawson0 -
Would you rate-control Googlebot? How much crawling is too much crawling?
One of our sites is very large - over 500M pages. Google has indexed 1/8th of the site - and they tend to crawl between 800k and 1M pages per day. A few times a year, Google will significantly increase their crawl rate - overnight hitting 2M pages per day or more. This creates big problems for us, because at 1M pages per day Google is consuming 70% of our API capacity, and the API overall is at 90% capacity. At 2M pages per day, 20% of our page requests are 500 errors. I've lobbied for an investment / overhaul of the API configuration to allow for more Google bandwidth without compromising user experience. My tech team counters that it's a wasted investment - as Google will crawl to our capacity whatever that capacity is. Questions to Enterprise SEOs: *Is there any validity to the tech team's claim? I thought Google's crawl rate was based on a combination of PageRank and the frequency of page updates. This indicates there is some upper limit - which we perhaps haven't reached - but which would stabilize once reached. *We've asked Google to rate-limit our crawl rate in the past. Is that harmful? I've always looked at a robust crawl rate as a good problem to have. Is 1.5M Googlebot API calls a day desirable, or something any reasonable Enterprise SEO would seek to throttle back? *What about setting a longer refresh rate in the sitemaps? Would that reduce the daily crawl demand? We could set increase it to a month, but at 500M pages Google could still have a ball at the 2M pages/day rate. Thanks
Intermediate & Advanced SEO | | lzhao0 -
If a website Uses <select>to dropdown some choices, will Google see every option as Content Or Hyperlink?</select>
If a website Uses <select> to dropdown some choices, will Google see every option as Content Or Hyperlink?</select>
Intermediate & Advanced SEO | | Zanox0 -
We have two different websites with the same products and information, will that hurt our rankings?
We have two different domains, one for the UK and the other for the US, they have the exact same products, categories and information. (the information is almost the same in 400 products) We know that Google could recognize that as duplicate content, but will that actually hurt our rankings in both sites? Is it better if we create two completely different versions of the content on those pages?
Intermediate & Advanced SEO | | DoitWiser0