Go to Settings then Apps and then there should be the list of installed apps. Choose one of the installers and then uninstall. This happens to change from version to version so it may not be the exact steps.
The Android Market is sometimes behaving strangely. I don't have any advice how to handle stuck purchases. But you need not worry that you only have one chance to download the app. After payment is done you can download it later at any time.
I'll move this topic to the new "Desire S" forum.