It is well known from a lot of scientific research that people are generally triggered far more greatly by losing something than they are excited by gaining something. This is part of why downvots are so contentious and potentially problematic. From a PR perspective their use has been a nightmare.
Ultimately, the votes are sometimes used as a form of Operant conditioning - which then taps in to all of the similar patterns people have experienced in their lives - some of which can be horiffic and evil. So the whole topic requires far more wisdom and understanding than is typically applied by people who view it merely as a weapon to be used in whatever way they like.