In theory it's true, but the principal of having others rule over you will still remain, no matter how small the government. And this principal is the very root of all our issues today, unfortunately. The thing we need is localized, strictly voluntary "governance". For eg. I think border patrolling is actually very important, but it should be handled by local militias along the border, not any centralized power that has all kinds of shady interests in letting everyone into their country while hiding away from it's consequences in secluded and guarded areas.