Can we say ‘thank you’ while refusing an offer?

What is the right way to refuse or reject an offer? Can we say thank you – like – Thank you. I will not be able to come along with you.