GoogleBot still crawling HTTP/1.1 years after website moved to HTTP/2

[AKCAC]

Whole website moved to https://www. HTTP/2 version 3 years ago.

When we review log files, it is clear that - for the home page - GoogleBot continues to only access via HTTP/1.1 protocol

• Robots file is correct (simply allowing all and referring to https://www. sitemap

• Sitemap is referencing https://www. pages including homepage

• Hosting provider has confirmed server is correctly configured to support HTTP/2 and provided evidence of accessing via HTTP/2 working

• 301 redirects set up for non-secure and non-www versions of website all to https://www. version

• Not using a CDN or proxy

• GSC reports home page as correctly indexed (with https://www. version canonicalised) but does still have the non-secure version of website as the referring page in the Discovery section. GSC also reports homepage as being crawled every day or so.

Totally understand it can take time to update index, but we are at a complete loss to understand why GoogleBot continues to only go through HTTP/1.1 version not 2

Possibly related issue - and of course what is causing concern - is that new pages of site seem to index and perform well in SERP ... except home page. This never makes it to page 1 (other than for brand name) despite rating multiples higher in terms of content, speed etc than other pages which still get indexed in preference to home page.

Any thoughts, further tests, ideas, direction or anything will be much appreciated!

[UmerIdrisi1]

Quoting here, to ask again, why this is happening with out pages too? is Google going crazy or what?

@James-Avery said in GoogleBot still crawling HTTP/1.1 years after website moved to HTTP/2:

@AKCAC said in GoogleBot still crawling HTTP/1.1 years after website moved to HTTP/2:

Whole website moved to https://www. HTTP/2 version 3 years ago.

When we review log files, it is clear that - for the home page - GoogleBot continues to only access via HTTP/1.1 protocol

• Robots file is correct (simply allowing all and referring to https://www. sitemap

• Sitemap is referencing https://www. pages including homepage

• Hosting provider has confirmed server is correctly configured to support HTTP/2 and provided evidence of accessing via HTTP/2 working

• 301 redirects set up for non-secure and non-www versions of website all to https://www. version

• Not using a CDN or proxy

• GSC reports home page as correctly indexed (with https://www. version canonicalised) but does still have the non-secure version of website as the referring page in the Discovery section. GSC also reports homepage as being crawled every day or so.

Totally understand it can take time to update such as our page at backwards 3 index, but we are at a complete loss to understand why GoogleBot continues to only go through HTTP/1.1 version not 2

Possibly related issue - and of course what is causing concern - is that new pages of site seem to index and perform well in SERP ... except home page. This never makes it to page 1 (other than for brand name) despite rating multiples higher in terms of content, speed etc than other pages which still get indexed in preference to home page.

Any thoughts, further tests, ideas, direction or anything will be much appreciated!

First off, it's great that your entire website made the transition to HTTPS and HTTP/2 three years ago. That's definitely a step in the right direction for performance and security.

Since your hosting provider has confirmed that the server is configured correctly for HTTP/2 and you've got the 301 redirects set up properly, it's puzzling why GoogleBot is still sticking to HTTP/1.1 for accessing the homepage. One thing you might want to double-check is if there are any specific directives in your server configuration that could be affecting how GoogleBot accesses your site. Sometimes, even seemingly minor configurations can have unintended consequences.

Regarding the non-secure version of your website still showing up in the Discovery section of Google Search Console (GSC), despite the homepage being correctly indexed with the HTTPS version, it could be a matter of Google's index taking some time to catch up. However, it's worth investigating further to ensure there aren't any lingering issues causing this discrepancy.

As for the home page not ranking as well in SERPs compared to other pages, despite having better content and speed, this could be due to a variety of factors. It's possible that Google's algorithms are prioritizing other pages for certain keywords or that there are specific technical issues with the homepage that are affecting its visibility.

In terms of next steps, I'd recommend continuing to monitor the situation closely and perhaps reaching out to Google's support team for further assistance. They may be able to provide additional insights or suggestions for resolving these issues.

Overall, it sounds like you've done a thorough job of troubleshooting so far, but sometimes these technical SEO mysteries require a bit of persistence to unravel. Keep at it, and hopefully, you'll be able to get to the bottom of these issues soon!

[Peter_Cox]

@john1408 said in GoogleBot still crawling HTTP/1.1 years after website moved to HTTP/2:

@AKCAC said in GoogleBot still crawling HTTP/1.1 years after website moved to HTTP/2:

Whole website moved to https://www. HTTP/2 version 3 years ago.

When we review log files, it is clear that - for the home page - GoogleBot continues to only access via HTTP/1.1 protocol

• Robots file is correct (simply allowing all and referring to https://www. sitemap

• Sitemap is referencing https://www. pages including homepage

• Hosting provider has confirmed server is correctly configured to support HTTP/2 and provided evidence of accessing via HTTP/2 working

• 301 redirects set up for non-secure and non-www versions of website all to https://www. version

• Not using a CDN or proxy

• GSC reports home page as correctly indexed (with https://www. version canonicalised) but does still have the non-secure version of website as the referring page in the Discovery section. GSC also reports homepage as being crawled every day or so.

Totally understand it can take time to update index, but we are at a complete loss to understand why GoogleBot continues to only go through HTTP/1.1 version not 2

Possibly related issue - and of course what is causing concern - is that new pages of site seem to index and perform well in SERP ... except home page. This never makes it to page 1 (other than for brand name) despite rating multiples higher in terms of content, speed etc than other pages which still get indexed in preference to home page.

Any thoughts, further tests, ideas, direction or anything will be much appreciated!

It's baffling that GoogleBot persists with HTTP/1.1 for the homepage despite proper setup. Consider exploring Google Search Console further for indexing insights, and reach out to Google Support for assistance in resolving this unusual behavior.

[James-Avery]

@AKCAC said in GoogleBot still crawling HTTP/1.1 years after website moved to HTTP/2:

Whole website moved to https://www. HTTP/2 version 3 years ago.

When we review log files, it is clear that - for the home page - GoogleBot continues to only access via HTTP/1.1 protocol

• Robots file is correct (simply allowing all and referring to https://www. sitemap

• Sitemap is referencing https://www. pages including homepage

• Hosting provider has confirmed server is correctly configured to support HTTP/2 and provided evidence of accessing via HTTP/2 working

• 301 redirects set up for non-secure and non-www versions of website all to https://www. version

• Not using a CDN or proxy

• GSC reports home page as correctly indexed (with https://www. version canonicalised) but does still have the non-secure version of website as the referring page in the Discovery section. GSC also reports homepage as being crawled every day or so.

Totally understand it can take time to update backwards 3 index, but we are at a complete loss to understand why GoogleBot continues to only go through HTTP/1.1 version not 2

Possibly related issue - and of course what is causing concern - is that new pages of site seem to index and perform well in SERP ... except home page. This never makes it to page 1 (other than for brand name) despite rating multiples higher in terms of content, speed etc than other pages which still get indexed in preference to home page.

Any thoughts, further tests, ideas, direction or anything will be much appreciated!

First off, it's great that your entire website made the transition to HTTPS and HTTP/2 three years ago. That's definitely a step in the right direction for performance and security.

Since your hosting provider has confirmed that the server is configured correctly for HTTP/2 and you've got the 301 redirects set up properly, it's puzzling why GoogleBot is still sticking to HTTP/1.1 for accessing the homepage. One thing you might want to double-check is if there are any specific directives in your server configuration that could be affecting how GoogleBot accesses your site. Sometimes, even seemingly minor configurations can have unintended consequences.

Regarding the non-secure version of your website still showing up in the Discovery section of Google Search Console (GSC), despite the homepage being correctly indexed with the HTTPS version, it could be a matter of Google's index taking some time to catch up. However, it's worth investigating further to ensure there aren't any lingering issues causing this discrepancy.

As for the home page not ranking as well in SERPs compared to other pages, despite having better content and speed, this could be due to a variety of factors. It's possible that Google's algorithms are prioritizing other pages for certain keywords or that there are specific technical issues with the homepage that are affecting its visibility.

In terms of next steps, I'd recommend continuing to monitor the situation closely and perhaps reaching out to Google's support team for further assistance. They may be able to provide additional insights or suggestions for resolving these issues.

Overall, it sounds like you've done a thorough job of troubleshooting so far, but sometimes these technical SEO mysteries require a bit of persistence to unravel. Keep at it, and hopefully, you'll be able to get to the bottom of these issues soon!

[john1408]

@AKCAC said in GoogleBot still crawling HTTP/1.1 years after website moved to HTTP/2:

Whole website moved to https://www. HTTP/2 version 3 years ago.
When we review log files, it is clear that - for the home page - GoogleBot continues to only access via HTTP/1.1 protocol

Robots file is correct (simply allowing all and referring to https://www. sitemap

Sitemap is referencing https://www. pages including homepage

Hosting provider has confirmed server is correctly configured to support HTTP/2 and provided evidence of accessing via HTTP/2 working

301 redirects set up for non-secure and non-www versions of website all to https://www. version

Not using a CDN or proxy

GSC reports home page as correctly indexed (with https://www. version canonicalised) but does still have the non-secure version of website as the referring page in the Discovery section. GSC also reports homepage as being crawled every day or so.

Totally understand it can take time to update index, but we are at a complete loss to understand why GoogleBot continues to only go through HTTP/1.1 version not 2
Possibly related issue - and of course what is causing concern - is that new pages of site seem to index and perform well in SERP ... except home page. This never makes it to page 1 (other than for brand name) despite rating multiples higher in terms of content, speed etc than other pages which still get indexed in preference to home page.
Any thoughts, further tests, ideas, direction or anything will be much appreciated!

t seems like you've taken several steps to ensure the correct protocol (HTTP/2) for your website, and it's puzzling that GoogleBot still accesses the home page via HTTP/1.1. A few additional suggestions:

Crawl Rate Settings: Check your Google Search Console (GSC) for crawl rate settings. Google might be intentionally crawling your site slowly.

Server Logs: Reanalyze server logs to confirm that GoogleBot is indeed accessing via HTTP/1.1 for the home page. This could help identify patterns or anomalies.

Mobile Usability: Ensure your home page is mobile-friendly. Google tends to prioritize mobile indexing.

Fetch and Render Tool: Use GSC's Fetch and Render tool to see how Google renders your home page. It might provide insights into how Google sees your page.

Structured Data and Markup: Ensure structured data and markup on your home page are correct and up-to-date.

Manual Submission: Consider manually requesting indexing for your home page through GSC.

Regarding the new pages performing well compared to the home page, it might be worth revisiting your on-page SEO elements and analyzing the competition for relevant keywords.

[john1408]

@AKCAC said in GoogleBot still crawling HTTP/1.1 years after website moved to HTTP/2:

Whole website moved to https://www. HTTP/2 version 3 years ago.

When we review log files, it is clear that - for the home page - GoogleBot continues to only access via HTTP/1.1 protocol

• Robots file is correct (simply allowing all and referring to https://www. sitemap

• Sitemap is referencing https://www. pages including homepage

• Hosting provider has confirmed server is correctly configured to support HTTP/2 and provided evidence of accessing via HTTP/2 working

• 301 redirects set up for non-secure and non-www versions of website all to https://www. version

• Not using a CDN or proxy

• GSC reports home page as correctly indexed (with https://www. version canonicalised) but does still have the non-secure version of website as the referring page in the Discovery section. GSC also reports homepage as being crawled every day or so.

Totally understand it can take time to update index, but we are at a complete loss to understand why GoogleBot continues to only go through HTTP/1.1 version not 2

Possibly related issue - and of course what is causing concern - is that new pages of site seem to index and perform well in SERP ... except home page. This never makes it to page 1 (other than for brand name) despite rating multiples higher in terms of content, speed etc than other pages which still get indexed in preference to home page.

Any thoughts, further tests, ideas, direction or anything will be much appreciated!