Every day when I use the DC Metro rail system, I think about this problem.
For those who don't know, the DC Metro is worse than you thought it ever could be. There's always too many people and not enough trains. Waiting 15 minutes for a train is not rare. Trains and rails catching on fire is so common that there's a twitter account specifically dedicated to announcing when it's on fire.
The list goes on and on, but you get my point, the DC Metro, like all public services, is terrible. Ask somebody vaguely associated with the American left how to fix it (saying democrat or liberal isn't usually very helpful, but you know who I'm talking about) and their answer will be some variation of "lack of funding," "the right people aren't running it," or it's somehow a byproduct of the Koch brothers' plot to destroy America. If you ask a regular Republican, they'll have some weird answer about how it needs to be run more like a business instead of a non-profit, but that it should remain a public entity. Ask more hardline conservatives and their answer will be simply "privatize it" (although this answer is gaining more ground with even less radical Republicans.
My intuition used to be the same. Privatizing a "public good" means that it's subject to a profit motive and will be run the best it possibly can be. This is true because entrepreneurs respond to incentives and will create the most value for consumers within the confines of natural scarcity.
The logic is sound, privately entities are inherently going to provide the best options for consumers when compared with government entities. The problem is that this argument doesn't consider the past.
If government solutions are not responding to what consumers value (which they can't) then why was any particular "public good" created in the first place? How do we know that these public goods would exist or exist in the same way in the private sector?
The solution then is not to privatize things, but eliminate them. If these things truly do create value then entrepreneurs will create them and they'll be better than ever. In the case of things like public transit, I suspect there is a demand and entrepreneurs would create a similar system but better in probably many ways. Things like the Post Office, which have private sector equivalents that are better in every way would just be gone. Private companies would move into the space the Post Office previously occupied and provide better service.
Of course, I'm just speculating and the only real way to know would be to eliminate them and see what things come up again and which are gone for good, not dictated by any one person or entity. It's subject to whatever creates the most value, as all things would be in the absence of a government.