The United States began withdrawing troops from Afghanistan, the US military said on Tuesday, taking a step forward on its peace deal with the Taliban while also praising Afghan President Ashraf Ghani’s promise to start releasing Taliban prisoners after he had delayed for over a week.

The US-Taliban deal signed on February 29 was touted as Washington’s effort to end 18 years of war in Afghanistan. The next crucial step was to be intra-Afghan talks in which all factions including the Taliban would negotiate a road map for their country’s future.

But Ghani and his main political rival, Abdullah Abdullah, were each sworn in as president in separate ceremonies on Monday. Abdullah and the elections complaints commission had charged fraud in last year’s vote. The duelling inaugurations have thrown plans for talks with the Taliban into chaos, although Ghani said Tuesday that he’d start putting together a negotiating team.

The disarray on the Afghan government side is indicative of the uphill task facing Washington’s peace envoy Zalmay Khalilzad as he tries to get Afghanistan’s bickering leadership to come together. In an early Tuesday tweet, Khalilzad said he hoped the two leaders can “come to an agreement on an inclusive and broadly accepted government. We will continue to assist.”

US military spokesman in Afghanistan Sonny Leggett said in a statement on Tuesday that the military had begun its “conditions-based reduction of forces to 8,600 over 135 days.”

Currently, the US has about 13,000 soldiers in Afghanistan — 8,000 of whom are involved in training and advising Afghanistan’s National Security Forces, while about 5,000 are involved in anti-terror operations and militarily supporting the Afghan army when they are requested.

Ghani had been dragging his feet on releasing some 5,000 Taliban prisoners, something agreed to in the US-Taliban deal. Ghani promised Monday to announce a decree to free the prisoners, after the US and a number of foreign dignitaries appeared to back his claim to the presidency by sending their representatives to his inauguration.

US Secretary of State Mike Pompeo released a statement Monday saying, “We also welcome President Ghani’s announcement that he will issue a decree March 10 on Taliban prisoner release.”

Taliban officials said late Monday that a flurry of biometric identifications were being conducted on Taliban prisoners, hinting at a mass release, according to prisoners currently in lockup. The Taliban officials spoke on condition of anonymity because they weren’t authorised to talk to the media.

Pompeo also said he “strongly opposed” the establishment of a parallel government in Kabul, despite the early signs of one emerging. Abdullah had quickly sent his vice-presidents to occupy the official offices on Monday, ahead of Ghani’s plan to send his vice presidents to their offices Tuesday.

Pompeo warned against “any use of force to resolve political differences.” Both candidates — but particularly Abdullah — are backed by warlords with heavily armed militias, underscoring fears they could use force to back their candidate.

The US has said its partial troop withdrawal over an 18-month period provided for in the deal will be linked to the Taliban keeping their promises to help fight terror in Afghanistan, but not to the success of talks between the Taliban and the Afghan government.

On the weekend, Taliban spokesman Zabihullah Mujahed said the insurgent group was committed to their agreement with the United States and called on Washington to do its part to make sure their prisoners were freed.

The militant Islamic State claimed responsibility for a rocket attack that took place during Ghani’s inauguration ceremony. IS also claimed a brutal attack last week on a gathering of minority Shiites that killed 32 and injured scores more. The US in reaching its deal with the Taliban said they expected the Taliban, which has been battling Afghanistan’s IS affiliate, to further aid in the effort to defeat IS.