It won't matter much coz you can't gain back your Voting power by removing your vote.