This extremely flawed and irrational concept of "The consent of the governed" or the "social contract" is probably the main reason the idea of government and authority still persists, so it can't be debunked enough times (until the myth dies out, that is).
The "social contract" is viewed as a way of society agreeing upon a set of sensible rules that discourage us from going back to the stone age and behaving like wild savages. Who can disagree with that? No one, if that's what the "social contract" is really about, but it's not. There is a difference between society establishing basic fundamental rules, and forgoing that right to those in government. The usual reply to this is something similar to: "The job of elected officials is to represent the will of the majority, and therefore, society in general." What exactly makes this the case? The people can't force politicians to keep their promises or pass laws they want. Sure they can threaten to vote for another candidate next time, but that's still not forcing the politicians to do anything. It just sometimes acts as an incentive for them to listen to what their constituents want. This is highly ironic, given that lawmakers do force society to comply with their laws under threat of punishment. It's hard to see how the people are actually "in charge" here, because they're not.
Not to mention, voters don't and can't know every law the politicians they vote for will repeal and pass, so how can they consent to something they don't know will happen? If the "social contract" was a real, written contract, it would read something like this:
"By voting, you hereby consent to whatever the politicians repeal and enact as law, even if you disagree with their decisions."
A more blunt (but just as accurate) version would look similar to the following:
"By voting, you hereby consent to the politicians doing whatever they want, even if you disagree with what they do."
This brings up another point. Some argue that just by living under the jurisdiction of a government, you consent to being subject to the laws of that government, which is why they often say "If you don't like the laws, then you can leave!". If this is true, then the government doesn't need to grant you the right to vote to be legitimate. Since its legitimacy is supposedly derived from "the consent of the governed", and "the governed" are consenting to its laws simply by living there, the government has no obligation to allow the people to vote. Voting then becomes a privilege, just as it would be a privilege, not a right, to be permitted onto someone's private property.
Even many who believe that the ability of the people to vote is what makes a government legitimate also use the argument that a person can leave if he or she doesn't like the laws, usually missing the fact that this applies to them as well. By voting, they consent to putting the politicians in power and to what the politicians do while in office, so what right to they have to complain or protest? They "can just leave" or wait until the next election. Whether the candidate the person voted for loses is irrelevant, because voting is consenting to the system that puts the winning candidate in office.
In reality, nothing can make governments legitimate because nobody has the right to use force outside of self-defense, including the people in government. The labels of "government", "authority" and "law" don't change this fact because they don't change the act of using coercion itself, meaning the act is still immoral.