Am I being paranoid or is he using me?

Your obviously not happy about just being his F*** buddy, and he has already made it clear (by using this term) how he sees you. So, no I don't think you are being paranoid.