If you think that the fact that someone concludes there might be a case for government action of one kind or another is sufficient reason to dismiss their argument for that position, you are not a libertarian, you are a dogmatist.
If you reject an argument solely on the grounds that the conclusion it reaches is at odds with your priors, you have implicitly claimed there's no argument or evidence that convince you to abandon those priors. Therefore your beliefs are dogmas and you are a dogmatist.
And you're not helping the cause of liberty if that describes you.
This is not excommunicating people who believe in the NAP (non-aggression principle, for those out of the loop). Actually, I think that's a perfectly fine way to be a libertarian. It's fine to believe in the NAP but if you reject arguments against the NAP because they lead to gov't action and not because they are simply bad moral arguments, logically or empirically then you are a being dogmatic.
If one's libertarianism depends on the validity of the NAP, then arguments against the NAP should be refuted by arguments FOR the NAP, not the claim that the state is bad.
And just to be clear, even though my own libertarianism is not based on the NAP, I think folks who do based theirs that way are just as much libertarians as anyone else. But if you justify your libertarianism by the NAP and you are faced with arguments against the NAP, the proper way to respond is by pointing out the flaws in those anti-NAP arguments, and presenting counter-arguments for the NAP. The wrong way to respond is "those anti-NAP arguments must be wrong because they lead to a larger role for the evil state." That's what I mean by dogmatism. You might still be a libertarian, but you're an irrational and dogmatic one who is not helping matters.
After all, how many times have you been in a debate with someone, only to be frustrated by them saying your argument must be wrong because they know that free markets are bad? That's exactly how YOU sound to them.
Unsound and invalid arguments have to be determined by the structure of the arguments and the quality of the evidence, not by the conclusion they reach alone.