Pheasey and New Invention have been put forward to shut in the next two years under proposals which will see a total of eight closed to save more than £500,000.
But figures have shown that Pheasey based at the Collingwood Centre, Collingwood Drive, generated 32,947 visitors in 2103/2014 compared to 31,747 in the 12 months beforehand. While New Invention, in The Square, Willenhall, increased from 25,102 to 28,878 during the same period.
Pheasey councillor and Conservative group leader Mike Bird said: "The library closures are politically motivated, there is no doubt about it.
"It is right on the edge of the borough, people depend on the services the library delivers and there is an isolation factor that has not been taken into account."
But Walsall Council leader Labour's Sean Coughlan said no final decision on which community facilities will go had yet been made. And that the consultation with residents across the borough was under way.
"This is a consultation. No decision has been taken. We have used the same process that Councillor Bird has used over a period of many years now. We will listen to what people say before this council makes any decision. We will not set the budget on February next year.
"We want consultation to be as open and transparent as possible, which is something that the people of Walsall have never had previously. The criteria is exactly the same criteria that officers have used in the past regarding the closure of buildings, such as Willenhall Leisure Centre," Councillor Coughlan said.
Under the draft budget proposals five libraries Streetly, Beechdale, Pheasey, South Walsall in Delves, and Walsall Wood will shut next year. Three more in Blakenall, New Invention and Rushall will close the following year. The move will save the authority £565,722 to £385,092 in 2015/2016 and £180,630 in 2016/2017.
The plan is part of the council's bid to save £86 million over the next four years following a squeeze on Government funding.
Campaigners in Pheasey have attracted more than 1,000 signatures for a petition to save the facility in Collingwood Drive from shutting, while in Streetly residents have attracted more than 2,3000 names.
Willenhall Leisure Centre that was based in Bath Street shut four years ago when the town hall was controlled by the Conservatives as part of a budget saving policy to save £22m in the 2010/2011 financial year and up to £70m by 2014.