TL;DR
-
Many sites and CDN’s across the internet use SNI to help keep pages and resources secure
-
Right now, Moz Pro is unable to crawl sites that use SNI, affecting these customers (you will see an 804 HTTPS SSL error if you are affected)
-
This is being fixed, and we are almost ready to provide beta access to SNI customers
-
If you have an SNI site, are a Moz Pro customer, and wish to be a part of this beta - sign up here http://goo.gl/forms/LCvL9Ix8JDHfbAvr1
Detail
SNI (Server Name Identification) is an extension to the TLS security protocol that allows for multiple HTTPS websites to be served from the same IP address without requiring them all to use the same certificate. This was always possible using wildcard certificates, but these are often impractical, since they are restrictive, and require a webmaster to know all of the hostnames to be served at the time of requesting the certificate. With SNI, a webmaster can issue a single certificate and support multiple (and changing) hostnames.
SNI has become increasingly popular over the last couple of years thanks to:
-
Increase in mainstream browser support
-
Its flexibility for usage in CDNs, like Cloudflare
Unfortunately the Moz Pro web crawler has not kept up to date with this trend, and is at present unable to crawl sites that use SNI (the symptom is an 804 HTTPS SSL error). This affects customers with SNI hosted sites, who miss out on the value of Moz Pro’s Site Crawl tools. Whilst it might seem a simple fix to support SNI, our crawler software uses libraries that are unable to support the technology without a significant re-write.
However, I am happy to say that this re-write is in process, and almost ready for testing. Behind the scenes we have been working on a major overhaul of our crawl technology which you are going to start to see across many of our products. In fact, Moz Content adopted the new crawler this week, resulting in much faster crawls and support for SNI.
Moz Pro is not ready for full implementation yet. That said, we are pretty close to being able to beta test with customers. For that reason we would like to invite any customers with SNI sites into our closed Beta that will likely start some time in August.
Some notes about the beta:
-
Initially, it will be a little behind where the current crawl tools are in terms of features and configuration. That said, we will be iterating fast and beta customers will get automatic access to all new features
-
As with any beta there will be bugs and potential data outages - I want to be transparent about this up-front
-
We will be looking for feedback throughout the beta, and we would love to have our customers help shape the future of our software
-
You must be a paid Moz Pro subscriber (or in a free trial)
If you are interested please sign up here - http://goo.gl/forms/LCvL9Ix8JDHfbAvr1. Questions and comments welcome below!