Council leader Mike Bird had previously said work on the centres at Oak Park, Walsall Wood, and High Street, Bloxwich, could start within weeks if the authority gave the go-ahead.
But concerns over rising costs and ownership of land led to councillors voting against the developments at a meeting of the full council meeting.
Now Councillor Bird has revealed that the plans could go back to the design stage to resolve the land issue and cut costs.
The development had previously looked set to be scaled back once before. But two months ago the council's cabinet voted to go ahead with building two new centres.
A motion to get the cost of the project included in the next year's budget was opposed at ameeting of the council.
Councillor Bird said he feared the plans would have to go through several rounds of scrutiny meetings and discussions to get passed.
And he said it may be easier for new plans to be drawn up.
He said today: "I think the plans have been stopped in their tracks. By the time we go through the due process I think it is best if we just start all over again.
"We want two new facilities to be built, but this vote may now mean that we have to go back to the drawing board. I do admit that the costs are huge and maybe we can only afford a Mini instead of the Rolls-Royce we were planning for."
The total cost of the project would be £24 million – with £2m coming from a government grant. The council is borrowing the rest of the money and will shell out £1.39 million a year in repayments – including interest – for the next 25 years.
And with running costs, it will spend £1.75m a year in total.
The council will not start paying any money back until 2015 when the building work is finished.
The Miners' Trust charity must agree to sell some of the land at Bloxwich and Leamore Recreation Ground, known as Leamore Park, which forms part of the proposal for Bloxwich Leisure Centre, in High Street.
But it emerged that the Charity Commission had not yet given its blessing because the sale of the land has not yet been resolved.