Members of the Chase Independents and Greens are no longer serving on the cabinet at Cannock Chase District Council, leaving Labour in a potentially precarious position.
Labour is the largest party on the council but is outnumbered by councillors from all the opposition parties.
Paul Dadge, the chair of newly-formed Chase Independents said the group was 'removed' from the cabinet.
Labour council leader George Adamson, however, said Councillor Paul Woodhead, of the Chase Independents, threatened to pull out last weekend.
Green Party councillor Andrea Muckley was also given the opportunity to stay on but chose not to, Councillor Adamson said.
The leader labelled the actions of the Chase Independents as "appalling" at a time the authority was attempting to deal with the fallout of the coronavirus pandemic.
He said while no formal coalition or confidence and supply arrangement had been agreed he was confident the Conservatives and Liberal Democrats would support the council at a time of crisis following conversations with them.
He said he did not believe opposition parties would gang up to try and topple the Labour administration.
Ex-Labour councillor Andrea Layton joined former Green Party councillors Paul Woodhead, Claire Wilkinson, Stuart Crabtree and Mandy Dunnett to form the Chase Independents.
It left Labour with 17 seats and the Tories with 14 at the council house.
Councillor Adamson said: "The timing was absolutely appalling in the middle of a pandemic. Cannock Chase district is facing a massive economic crisis. We are expected to be the fourth worst affected of the whole of the West Midlands.
"The Chase Independents chose to play political games at a time of crisis and I think it's appalling. We should be pulling together to get through the crisis."
He added: "I'm confident the rest of the council will take a more mature attitude than the Chase Independents have.
"There is no formal coalition and no formal guarantees but we want to work together."
Mr Dadge said Labour was "clearly frightened of change and the new ways of working represented by our Group".
He added: "We will continue our positive approach towards community engagement and keep fighting to bring the community closer to the working of the council. The Chase Independents and Green group are clear that the priorities of the council should be informed by the community, not the other way round."