I personally love https://sucuri.net to fix issues like this.
So you need to take two steps, the first one is you need to work out if someone is hacking into your site by using sucuri you can check. They will also do a clean up if they find something. To find the page they hacked into you can also look at where these links are pointing to.
The second part is you need to do a disavow of the links going to the hacked page. As this will also be hacked sites most likely or cheap links. Here is the link to do a disavow, you do this through search console.
Good luck and if you need anything else let me know.