Hi Aspirant
Yes, you are correct.
IP Address and hosting of your website also effect on ranking. In my situation I also faced so man times. However, there are some websites which changes doesn't affect ranking.
You need to check your Security and Manual action in Google Search Console of our website. Also Don't Forget to check coverage of the site
Thanks and have a good day
Ali
SEO Speciaist in elabhouse