yes, to do so you need to go into the survey template that you are using, make a copy and then modify it. You can delete the code for this button or you can put it into comments which will hide it.
How this is done, depends very much on the template you are using. In Bootswatch you'll probably find it in clearall_links.twig, if I am not mistaken.
I answer at the LimeSurvey forum in my spare time, I'm not a LimeSurvey GmbH employee.
No support via private message.