This is exactly how the system is designed though. The consensus on how much a post is worth is based on the sum of all the upvotes and downvotes at the end of 7 days. It is very similar to how ratings work on most other content platforms too. If a movie gets four people who give it five stars, and one who gives it one star - the movie will show a 4 star review. I don't think there is anyone who would say that the movie deserved five starts because of the four people who said it did. Most people would agree that based on the votes it had received, it's current rating is at 4 stars.