Southern and East African leaders have rejected plans to set up a pan-African government, as suggested by Libya's head of state Colonel Moamar Gaddafi.
Uganda's Yoweri Museveni says he backs economic integration but says Africa is too diverse for one government.
Senegal, however, backed the plans and says a breakaway group could be formed.
Ghana's Foreign Minister believes problems are inevitable but can be overcome as the European Union has done.
African leaders split on unity
An African Union summit in Ghana has overrun after leaders struggled to reach a compromise on moves to form a closer union.The three-day meeting has been dominated by calls for a so-called United States of Africa but while member nations agree on the goal of economic integration and eventual unity, most leaders have urged caution.
Muammar Gaddafi, the Libyan leader, and Abdoulaye Wade, the Senegalese president, lead a group of countries who have been pushing for the immediate creation of a federal state.
On Tuesday, Gaddafi called for a referendum to settle the issue.
"We ask all the heads of state to hold a referendum so that they will see that all the people want a United States of Africa," he said in a speech to the summit.
Wade also promoted the creation of a federal government when he spoke to journalists late on Monday.
"There is no salvation for Africa outside political unity. ... If we remain fragmented into little states, we will remain weak, politically weak," he said.
Asked about earlier Senegalese threats that a group of five or six states could forge ahead with federation, Wade said: "Theoretically, it is not excluded ... but I don't think we'll be going in that direction.
"If the conference as a whole makes progress towards a government that it calls a continental government , a union government ... that will create a basis that we can accept."
Other countries, including the big regional powers of Nigeria and South Africa, have called for a more gradual move towards greater union.
"In Uganda, we are not in favour of forming a continental government now," Yoweri Museveni, the Ugandan president, said.
He said that while economic integration was possible, people from different regions of Africa were incompatible politically and forcing them together would create tension.
Complex situation
"I salute the enthusiasm of those who advocate for continental government now. I however, do not want us to move from one mistake - Balkanisation - to another mistake of oversimplification of very complex situations," Museveni said.
Pakalitha Mosisili, Lesotho's prime minister, summed up the view of the moderates: "Even as we pursue this noble objective, we cannot ignore the factors that militate against it."
He said surrender of national sovereignty was a "tall order".
Haru Mutasa, Al Jazeera's correspondent in Ghana, said questions also remain over who will pay for a united Africa.
"Most African countries can't even fund themselves, so will they ask the World Bank to give them money, and if they do what strings will come attached to it," she said.