Currently no - but I do get what you're going for (and totally understand why lol).
Really the only "real" way I see it actually working is to transfer ownership to an account where the keys are burnt. That way moderation could be disabled (by first owner), then ownership transferred (to burnt owner) which would prevent moderation from ever being enabled again.
I suppose I could simulate that just by allowing the owner to essentially 'burn' their moderation powers with a custom operation and then just never allow it to be reenabled.
It's something I'm definitely thinking about!
RE: Would it be possible to remove moderation permissions?