I did that once, and I see nothing wrong with downvoting reactions. It is not a kindergarten where everyone is given praise and participation award. Sometimes negative feedback is warranted. If you upvote, why the opposite is wrong? Would you never scold a litterer?

Please remember to be polite.