Moussa Traore, who led Mali's first military coup in 1968, died at his home aged 83 in the capital Bamako on Tuesday, his family said.
Traore was the main instigator of a coup that overthrew Modibo Keita, the country's first post-independence president.
He became president the following year and ruled until 1991, when he was overthrown by a military takeover.
He is known for ruling with an iron fist but also for his diplomatic skills. As chair of the then Organisation of African Unity (now the African Union), he played a key role in the 1989 Senegal-Mauritania crisis and the Chadian-Libyan conflict, as well as Liberia's first civil war.
Traore reached a peace agreement in 1990 with Mali's Tuareg armed rebel groups after making significant concessions.
But in 1991, the soldiers he had sent to quell pro-democracy demonstrators turned against him and overthrew him in a bloody insurrection, which officially resulted in more than 200 dead and 1,000 wounded.
He was sentenced to death for "political crimes" in 1993 and, along with his wife, for economic crimes in 1999.
The sentences were commuted to life in jail, and Traore was pardoned in 2002.
In recent years, Traore was increasingly seen as an elder statesman, with politicians soliciting his advice.
Traore's death comes just four weeks after another putsch in Mali after rebel army officers overthrew president Ibrahim Boubacar Keita on August 18.