I haven't used my card for deposit but can I withdraw funds to the card?

No, you are not able to withdraw the funds to the card if you didn't use it for deposit. Card withdrawal can be applied to a customer who deposit with a card, and the maximum amount of card withdrawal is the amount deposited with that card.