The cash-strapped authority is mulling over plans to uproot Bilston Library from its home of 145 years as part of its cost cutting measures to transform libraries in the city.
But Wolverhampton South East's Labour MP Mr McFadden says 'no convincing case' has been made for the plans, which would see the library moved to Bilston Town Hall.
"Bilston Library has been housed in the Craft Gallery on Mount Pleasant for many years," he said.
"No one has ever raised its location as a problem with me.
"I know the council is under huge financial pressure, but I don’t believe there is local support for moving it or that a convincing case for moving it has been made.
"I think the priority should be a plan to get more use for the whole Bilston Craft Galley site including the library, the exhibition space and the beautiful garden behind the building."
Bilston Craft Gallery currently houses the library, exhibition space and children's activity centre Craft Play.
Wolverhampton council has delayed a decision on the library's future pending the result of a detailed feasibility study.
A public consultation saw 635 people have their say on the issue, with 167 people backing the library staying at its current home and 138 opting for it to move to the Town Hall.
A total of 26 people wanted to see it move to another location while 108 had no opinion and 196 left the answer box blank.
Mr McFadden has written to Wolverhampton council's managing director Keith Ireland urging him to keep the library at its current base on Mount Pleasant.
"My fear is that if the library moves without any convincing plan for future use of the craft gallery building in place, it will be difficult to sustain and the area could end up losing this important public asset," he wrote.
"I would not want that to happen. I believe it is very important that this building and its grounds remain available to the public in Bilston.
"I would like to see a plan put in place based on increasing the use of this building and attracting more people to it."
He has also called for increased opening hours for city libraries, some of which have seen their opening hours slashed to just 15 hours a week.