unless you increase that amount which would probably take away rewards from authors and curators

That's what I was referring to. Making the number of consensus witnesses higher does not directly cause the witness portion of the mint pool (I just made that term up, I'm sure there's a proper word for it...) to increase.

so each witness would earn less

Not really. The consensus witnesses would earn less, but assuming only the consensus witness number to change, the witness rewards would only be distributed "better" among all the witnesses. While increasing the number of consensus witnesses would probably add a little more decentrality at the cost of a little harder organisation when hardforking etc. I don't think that the major problem lies there. I think that the big problem is that a single account can vote enough witnesses to reach consensus all by itself. I would much rather see the number of witness votes limited to < Consensus threshold. While that would enable a smaller part of the stake to block any hardfork, I think that's okay. For example the US constitution can also only be changed if 2/3 agree on it, and just like that in Austria as well. It's not uncommon to have a minority able to block major changes to the fundamentals of a system.