Welcome to the Q&A Forum

Hi, OP here (on a different Work account!)

If they upgrade to the pro level, CF SSL certs should now auto fallback for non SNI requests.

Not ideal, and Moz still need to fix the underlaying issue with the out of date version of Python that Rogerbot is written in... but it should provide a fix.

Hmm ok, thats something to go on then. Moz Support couldn't crawl when i asked them to check, and gave me nothing conclusive to work with other than 'check your logs' which were completely empty (no hits in access log or against CF firewall logs)

Guessing the crawltest is just broken at the moment then...

I'll setup a campaign and see if it works. Will update when i know one way or another.

Thanks!

Hi Lee,

Thats interesting.. Can you use the Crawl test tool on them? or does that fail?

Currently i have no campaignes setup as i just wanted to confirm if i could crawl or not. The crawl test tool fails for me with both SSL and non SSL sites on CF pro accounts...

Hi Lee,

OP here (Work account rather than personal),

Are the ones not being crawled behind SSL? As far as we can make out, Cloudflare should fall over to support non SNI crawlers for all CF Pro and above accounts. It seem however that RogerBot is spoofing a modern UA but not actually supporting modern SSL. The end result is that CF doesnt fall back properly and Moz can't crawl.

As i post this, im waiting on a Moz support response to a non SSL site behind CF thats not being crawled. Will post up if we work out what that is.

Hi Lee,

OP here (Work account rather than personal),

Are the ones not being crawled behind SSL? As far as we can make out, Cloudflare should fall over to support non SNI crawlers for all CF Pro and above accounts. It seem however that RogerBot is spoofing a modern UA but not actually supporting modern SSL. The end result is that CF doesnt fall back properly and Moz can't crawl.

As i post this, im waiting on a Moz support response to a non SSL site behind CF thats not being crawled. Will post up if we work out what that is.

Hmm ok, thats something to go on then. Moz Support couldn't crawl when i asked them to check, and gave me nothing conclusive to work with other than 'check your logs' which were completely empty (no hits in access log or against CF firewall logs)

Guessing the crawltest is just broken at the moment then...

I'll setup a campaign and see if it works. Will update when i know one way or another.

Thanks!