More than 22,000 Afghan families have fled from their homes to escape fighting in the former Taliban bastion of Kandahar, officials said on Sunday, as authorities arrested four suspected insurgents over this week's rocket attack on Kabul.

Since early May, violence has surged across several provinces including in Kandahar after the insurgents launched a sweeping offensive just days after the US-led foreign forces began their final withdrawal.

The Taliban's deadly assault has seen the insurgents capture scores of districts, border crossings and encircle several provincial capitals.

“The fighting has displaced 22,000 families in the past one month in Kandahar,” Dost Mohammad Daryab, head of the provincial refugee department, told AFP.

“They have all moved from the volatile districts of the city to safer areas.”

Fighting continued on the outskirts of Kandahar city on Sunday.

“The negligence of some security forces, especially the police, has made way for the Taliban to come that close,” Lalai Dastageeri, deputy governor of Kandahar province, told AFP.

“We are now trying to organise our security forces.”

Local authorities had set up four camps for the displaced people who were estimated to be about 154,000.

Kandahar resident Hafiz Mohammad Akbar said his house had been taken over by the Taliban after he fled.

“They forced us to leave ... I am now living with my 20-member family in a compound with no toilet,” said Akbar.

Fears of fighting to increase

Residents expressed concerns that the fighting might increase in the days ahead.

“If they really want to fight, they should go to a desert and fight, not destroy the city,” said Khan Mohammad, who moved to a camp with his family.

“Even if they win, they can't rule a ghost town.”

Kandahar, with its 650,000 inhabitants, is the second-largest city in Afghanistan after Kabul.

The southern province was the epi-centre of the Taliban's regime when they ruled Afghanistan between 1996 to 2001.

Ousted from power in a US-led invasion in 2001 after the September 11 attacks, the Taliban have spearheaded a deadly insurgency that continues to this day.

Their latest offensive launched in early May has seen the group take control of half of the country's about 400 districts.

Earlier this week, the chairman of the US joint chiefs of staff General Mark Milley said the Taliban appear to have “strategic momentum” on the battlefield.

But global rights group Human Rights Watch said there were reports that the Taliban were committing atrocities against civilians in areas they had captured, including in the town of Spin Boldak near the border with Pakistan they captured earlier this month.

“Taliban leaders have denied responsibility for any abuses, but growing evidence of expulsions, arbitrary detentions, and killings in areas under their control are raising fears among the population,” said Patricia Grossman, associate Asia director at HRW said in a statement.

The authorities meanwhile announced they had arrested four men they said belonged to the Taliban, accusing them of carrying out this week's rocket attack on Kabul.

“A Taliban commander, Momin, along with his three other men, have been arrested. They all belong to the Taliban group,” ministry spokesman Mirwais Stanikzai told reporters in a video message.

At least three rockets landed near the palace on Tuesday as President Ashraf Ghani and his top officials performed outdoor prayers to mark the start of Eidul Azha.

The attack was however claimed by the militant Islamic State group.