The poll is taking place in eight other countries that have substantial numbers of southern Sudanese.
Among the first to exercise the choice was Salva Kiir, the president of southern Sudan.
Addressing the crowd after he voted, Kiir paid tribute to John Garang, who had led the south to a 2005 peace deal that ended a 22-year war with the north. Garang died in a helicopter crash soon after the deal was signed.
"Dr. John Garang, and those that died with him in the struggle, are here with us today and we hope that they did not die in vain," Kiir said.
"This is the moment you have been waiting for," he told the crowd, asking them to have patience.
"Even if you cannot vote today, you can vote tomorrow."
Kiir said that in no way should the vote be jeopardised, telling the security forces to protect all the people, especially people from the north.
A total of 3.9 million southerners have registered for the self-determination vote that may lead to the partition of Africa's largest country.
Egyptians vote in historic election23/05
Africa Day 2012 - a moment for reflection and…22/05
South Africa's African agenda21/05
Women struggle to rinse hunger, poverty stains21/05
Climate Climate change affects migratory birds…21/05
Ghana: Foreign retailers cited for currency…18/05
Kenya: Community radio brings succour to…18/05
Veld fires 'flame' Zimbabwe's…16/05
