I think all downvotes should have a remark as to why. Esamples:

• Downvoted for Plagiarism
• Downvoted for excessive reward from pumpmeup account/bid bot
• Downvoted for excessive self voting
• Donwvoted for Vote for me I vote for you group.

Or similar short easy to understand reasons. I think the excessive rewards do need to be pointed out, not just Downvoted for rewards, the Author has no control in many cases as to who and how much a person votes on them. By pointing out the excessive upvoters instead of the post Author that may take some of the sting out, and at the same time let the excessive upvoter know their vote action was above and beyond what the downvoter felt was a justified reward.

I still think most of the issue with downvotes is that many times the community is not allowed to voice their opinion on the actual downvote. There are a lot of minuscule 15HP or less accounts that just go out downvoting randomly, by creating a system where a downvote needs to have a comment would solve that, even if it is just an auto comment.