I don't think so it is badly impact for SEO. It is an external script loading async on your website but there is a thing which might be considered as bad.  Sometime users who reviewed on google has private profiles which makes their profile picture 404 not found in google reviews plugin. I have googled it but didn't find any solution. So, i have come up with the lazy load solution but that's not permalink. It is just delayed loading the images which render the page quickly. Without using the lazy load the 404 not found links might take extra seconds for requests.  I used this plugin on my website  http://t20slam.com to show the reviews on my event but removed it because of 404 images errors. Other than this i think it is ok.