It's my advice to avoid starting a war.

Downvoting to get revenge for being flagged may feel satisfying, but it is also a good way to get flagged some more.

Personally I don't think flaggers should get special protections, but if the people getting flagged just revenge and don't change their posting and voting behaviors, they will just be flagged again.