The monarch also replaced the heads of the ground forces and air defences, as well as senior officials at the interior ministry, in a series of late-night royal decrees.
No official reason was given for the sweeping overhaul, but it comes as the Saudi-led coalition's military intervention in Yemen against Iran-aligned Huthi rebels nears the end of its third year.
"Termination of the services of General Abdul Rahman bin Saleh al-Bunyan, Chief of Staff," the official Saudi Press Agency (SPA) said, adding that Fayyad al-Ruwaili had been appointed as his replacement.
Al-Bunyan's sacking came after he inaugurated a major exhibition this week by the Saudi Arabian Military Industries (SAMI), the state-owned defence company, which illustrates ongoing efforts to reform the sector.
"These Saudi royal decrees appear to be part of the kingdom's national defence strategy," Theodore Karasik, a senior advisor at the consultancy Gulf States Analytics, told AFP.
"A military transformation is under way in Saudi Arabia. The changes come on the heels of the SAMI exhibition, which is a critical part of the Prince Mohammed's reform plan to create an indigenous defence program."
Crown Prince Mohammed bin Salman, the heir to the throne and the son of the monarch, is the country's defence minister and has been consolidating his grip on power in recent months.
The changes come in the midst of an ever-worsening conflict in Yemen.
A Saudi-led coalition supporting Yemen's government has been fighting the Huthis since 2015 in a conflict that has led to what the United Nations describes as the world's worst humanitarian crisis.
More than 9,200 people have been killed in the conflict and another nearly 2,200 Yemenis have died of cholera amid deteriorating sanitation conditions, according to the World Health Organization.
Separately, a series of civil appointments were announced in other decrees late Monday.
In an unprecedented announcement, a Saudi woman, Tamadar Bint Yousef al-Ramah was appointed the deputy minister of labour and social development.
And Prince Turki bin Talal the brother of billionaire Prince Al-Waleed bin Talal, was appointed deputy governor of the southern Assir Province.
Prince Al-Waleed, dubbed the Warren Buffett of Saudi Arabia, was among princes, ministers and tycoons detained in Riyadh's luxury Ritz-Carlton hotel in an unprecedented crackdown on what the government called elite corruption.