I fully support this change. That's actually the way I use the flag right now. Anything I dislike I will downvote, and no matter what the 'flag' says when I click on it, I can click it whenever I want.

I also believe the downvote should be part of the curation algorithm. Users who downvote should have more power on their upvotes. A typical user should have about a 80% upvote/downvote ratio as is the case on reddit (a finely tuned website that regularly pushes good content to the front page).