Hey Hive account@holisticlia!
If you are looking at the value of your SP in US$ that is at the bottom of the wallet, that will go up and down depending on the market price for Steem$. :)
No one can actually change your steem $ etc once its in your wallet. Downvoting will only reduce the SP award on a post BEFORE its been paid out... so within the first 7 days of posting. ;)

Hope that helps!