Defence Minister Tariq Shah Bahrami said on Monday "about 100 security forces" were killed in the intense fighting, as well "between 20 and 30 civilians".
Col. Fared Mashal, the province's police chief, said the majority of the insurgents fighting in Ghazni are foreigners, including Pakistanis and Chechens.
Food supplies in the city were "reportedly running low", he added.
"The city became so risky", the 60-year-old Mustafa told The Associated Press while stopped briefly at a checkpoint where police searched for wounded Taliban fighters.
Screen grab from AFPTV video taken on August 10, 2018 shows smoke rising after Taliban militants launched an attack on the Afghan provincial capital of Ghazni.
Roads in and out of the city have been damaged and obstructed by Taliban forces to prevent Afghan reinforcements arriving but residents who escaped the violence on foot have described seeing dead bodies in the street and buildings on fire.
The United States have also sent military advisers to aid Afghan forces and are conducting air strikes to support Afghan troops on the ground.
The Taliban swiftly responded, saying the government's claims were "baseless" and that talks were "under way for their surrender". "We do not know whether they went back to Ghazni, came to Kabul or have been taken away by Taliban", one Ghazni resident in Kabul said.
Sporadic clashes are continuing, O'Donnell said.
Barmak further added that the Afghan forces have full control over all governmental institutions and compounds including the central prison of Ghazni.
The Ghazni fighting adds to an increasingly fevered political atmosphere ahead of parliamentary elections in October, which have faced widespread concerns over potential security threats from both the Taliban and other armed groups. He said the casualty figures are not yet definite and that the numbers might change.
Bahrami did not give a breakdown of the casualties but Wais Ahmad Barmak, the interior minister, said that about 70 of the dead security forces were police officers.
Najib Danish, the Interior Ministry's spokesman, said reinforcements have been sent Ghazni and were trying to clear it of the Taliban. He said the Taliban had also suffered heavy casualties including about 50 fighters killed by an air strike on Sunday.
Mustafa's wife Razia said they had no food, water or and electricity for the past four days.
"There were so many dead bodies under the bridges, at the side of roads and under the destroyed houses", she said.
Another witness, Abdul Wasih, 59, said his son was wounded in the leg by a mortar attack Sunday that killed five of his close relatives. They also said the hospital is running out of medicine. Bodies wrapped in sheets lay in open wooden coffins in the hospital.
"Medication at the main hospital is reportedly becoming scarce and people are unable to safely bring casualties for treatment", Dr. Rik Peeperkorn, the acting U.N. Humanitarian Coordinator for Afghanistan, said in a statement. The attack began on Friday, with insurgents infiltrating people's homes and slipping out into the night to attack Afghan forces in Ghazni. "The whole of Ghazni province is on fire". Mohammad said locals were coming out of Ghazni and military convoys were heading in.
The United States and North Atlantic Treaty Organisation formally concluded their combat mission in Afghanistan at the end of 2014, but have since then repeatedly come to the aid of Afghan forces as they struggle to combat the resurgent Taliban.
Afghan President Ashraf Ghani is said to be considering a cease-fire offer to the Taliban for the upcoming Muslim holiday of Eid al-Adha, which starts August 21.
Just last month, representatives of the Taliban held the first direct talks with USA officials, part of a push by the Trump administration to end the 17-year conflict.
The Taliban say they met with Alice Wells, the top United States diplomat for South Asia, in Qatar last month for preliminary talks. Washington neither confirmed nor denied the meeting, but acknowledged Wells was in Qatar, where the Taliban maintain an office.
"They (Taliban) steal people's belongings".
In the meantime, the New York Times has reported that USA military commanders are urging Afghan security forces to consolidate their checkpoints by withdrawing from sparsely populated areas.
"The real terrorist threat is Islamic State (IS, formerly ISIS/ISIL), which has launched all these attacks, whether it be in Syria, in Iraq and even in Europe", Ali Rizk told RT.
"U.S. advisors are assisting the Afghan forces and US airpower has delivered decisive blows to the Taliban, killing more than 140 since, August 10".