Thousands of people have signed petitions to ensure Streetly and Pheasey facilities are retained after being identified to shut under cost-cutting measures.
A motion asking Walsall Council to keep them open and find savings from elsewhere was rejected by the Labour-run authority, while they go through the consultation process.
There are eight libraries across the borough which are under threat of closure in the next years in a bid to save more than £500,000. Final plans will go before the council next month.
Pheasey councillor Mike Bird, leader of the opposition Conservatives vowed: "The fight doesn't end here" after the motion was defeated by 30 votes to 26 at Full Council this week.
More than 3,000 people have signed petitions to keep Streetly open and more than 1,500 in Pheasey, which forced the matter to be debated by councillors in the Chamber.
Councillor Bird spoke in support of both libraries. He said Streetly was used by people looking to use computers to seek employment and there could be an impact on nearby traders with reduced amount of customers.
He added that Pheasey was on the periphery of the borough and people would have to travel further afield for an alternative.
Streetly councillor Gary Clarke said the facility provides services for adults and children in the area.
"There are people who are passionate about saving Streetly library," he said. "They realise how important the library is to the education of people in Streetly."
Pheasey councillor Chris Towe said it would be a "tragedy" for his ward and called for both library services to be kept. "It is a valued part of the community in both areas," he said.
Liberal Democrats group leader Ian Shires said: "All libraries tend to be the focus of a community and because of the footfall it has an effect on the local economy."
Leisure chief, councillor Khizar Hussain, said the only reason it was on the table and consulted on was due to Government funding cuts.
The library plan is part of the council's bid to save £86 million over the next four years.
Others earmarked to shut in this year's budget are Beechdale, South Walsall in Delves, and Walsall Wood.
Then three others in Blakenall, New Invention and Rushall in the following 12 months.