At the most basic level every human being has 2 fundamental rights, with a third included to preserve the first two. In a just society, these rights must not be infringed without due process. That due process must include trial and judgment by a jury of ones peers. It is from these fundamental rights, that all other rights, or any restrictions, may be logically derived.
The first basic human right, is the right to life. When I make the claim, that every human being has a right to life, I mean that they have a right that no other person shall: kill them, assault them, threaten them with violence, or impede their movements against their will. No more, and no less.
The second basic human right, is the right to private property. When I make the claim that every human being has the right to property, I mean that they have a right that no other person shall deprive them of their justly acquired property through: theft, vandalism, fraud, or coercion. No more, and no less. This can be partly derived from the right to life, and the idea of self ownership, such that a person has a right to the fruits of his labor, and therefore any property created by such labor, or justly acquired through voluntary trade.
If we are to grant that these two rights are fundamental and inviolable, then we must then also grant the right of every person to defend these rights from violation, in whatever manner they deem necessary. Therefore I claim a third right of every person, of self defense.
If we can agree that these rights are fundamental and inalienable, then we will be able to derive and justify any additional rights, as well as make clear any restrictions that must be placed on personal activities, through a process of logical deduction. Any activities which violate these rights are immoral and should be forbidden. Any activities which do not violate these rights must therefore be allowed.