The numbers are your reputation. If you get more upvotes, the number goes higher. If you get downvoted, it goes lower.