One important justification for having an armed citizenry is to keep the government from having a monopoly on the means of violence. If the government has that advantage over its citizens, so the logic goes, there is nothing to keep the government from using its armed advantage to displace democracy with authoritarian rule.
One of the major confounding arguments for giving the government the sole right to use force to settle disputes is that over the last several centuries the development of strong governments that were vested with the exclusive right to use violence has been the primary source of the reduction in violence in our everyday lives.
When governments were weak and did not have organized police forces (really a relatively recent invention) or standing armies (they had to raise armies as they planned military action), citizens were constantly threatened by a broad range of violent forces: There was a constant threat of armed brigands invading your farm, stealing your animals and stored food and money and assaulting, even killing your family. In addition, the ruling nobility with their private armed forces could do more or less whatever they wanted to you and yours. Outlaws, mostly unattached armed young men, prowled the roads and populated the woods. Within communities, disputes between people or families were often settled with violence that could trigger years of revenge killings as clans fought clans over grievances from the distant past. With weak, decentralized, or non-existent governments, people regularly settled conflicts and grievances by using violence themselves, individually or in groups. Religious zealots could mobilize group hysteria to hunt, torture, and kill heretics, sinners, witches, and people living unconventional lives.
Male-dominated, honor-based cultures led to contest for honor, status, dominance, and glory outside any legal structure. Violence was used to defend honor against perceived slights, insults, or challenges. The swelling and bruising of the male ego led to violent retaliation for perceived insults and the veneration of a man’s violent abilities and capacities. Think of the repeated stories in Hollywood’s version of the 19th century American West or the medieval clashes of knights or the Japanese Samurai defense of honor or the practice of dueling over perceived slights to honor that stretched over centuries into the 19th century. Mob violence in the United States stretched well into the 20th century imposing extra-legal controls over African American’s long after they were legally freed from slavery.
The point is that it was relatively strong governments that imposed “the rule of law” that set up institutions to settle grievances and punish law-breakers. That “rule of law” in effect gave the government the sole right to use deadly force except in very limited circumstances, primarily immediate self-defense. The result was a dramatic decline in violence where effective organized governments existed. In “failed states,” of course, the violent chaos continued and continues.
It was also true that such strong governments, having monopolized the use of violence, could oppress citizens in intolerable ways. How is the citizenry to fight back in that situation if it is not also armed and free to use violence to resist? The answer to that was taught to us by Gandhi in his non-violent struggle to end the British colonial control of his native India. Much of the European colonial empires disintegrated not because they were overpowered by indigenous armed force but because of the non-violent resistance of citizens who appealed to the human rights values shared by the citizens of the European colonizers.
The more recent disintegration of the Soviet Union and its puppet governments in Eastern Europe was also not the result of armed struggle. It was the result of the ongoing development of resistance within civil society in many of the countries that were part of the Soviet “evil empire.” The leadership of some of the nastiest of the Soviet Bloc countries was simply not willing to fire on its own citizens or the police forces and armies were not willing to do so. Again, non-violent resistance built around shared national values and human rights led to a largely non-violent collapse of a heavily armed totalitarian empire. One could go on. The decision of the white-dominated government of South Africa to free Nelson Mandela from decades of incarceration and negotiate a democratic transition to majority Black rule was not primarily based on an actual armed Black military uprising. Again it was the realization by the Whites that there was no legitimate justification that could be made for continued minority rule.
More recently the Arab Spring saw the fall of the Tunisian and Egyptian dictators to non-violent political protests. In Libya and Syria, where armed conflict supported by the flow of weapons from outside countries has dominated the struggle, a much bloodier conflict took place that is nowhere near ending.
Overall, it appears difficult to make the case that having a weak government and a heavily armed population that stands ready to take the law into its own hands lays the basis for a peaceful and prosperous state that values tolerance and respects human rights and individual differences. That has not been the historical experience.