I can safely assume that more than 50% of the authors and newly backed users will do that.
Right now they can also do it by using (or misusing) the flag. But they don't (at least most of them). Good authors have their reputation at stake which prevents them from misbehaving.
RE: How to fix the downvoting issue?