Paypal:
1. I don't have to give my CC details to yet another small business. This is the main thing. Both for security and because it's one less place to update when my card expires, etc.
2. Paypal is always approved by my bank. With a random hosting provider, I might have the "denied by...