I think we're arguing the same point with different semantics.
I don't like calling it a right because, in application, rights are given to you, and can therefore be taken away. While I love the idea of rights given to you by birth, that isn't very practical. The rights that are inaliable from birth can be argued in different political contexts, and the very existence of these rights can be argued as well.
The "Do No Harm" rule is nice but also impractical.
RE: Do You Own Your Thoughts? Prominent, Self-Proclaimed Libertarians Say "No."