Just 500 of around 3,700 items are being displayed in the town.
Thousands of items are not on show at the multi-million Walsall New Art Gallery and are being kept in storage.
The collection, which ranges from paintings, and drawings to ornaments includes Freud's portrait of Kitty, Van Gogh's 'Sorrow' and Monet's landscape.
But figures released under the Freedom of Information Act show although Walsall Council holds 3,700 works of art, only around 400 to 500 works are on display at any one time. Bosses have insisted that room is limited and rotating the collection keeps it 'dynamic'.
The £21 million Walsall New Art Gallery is considered a jewel in the crown for the borough, but keeping the large amount of items out of the public gaze has come under fire.
Blakenall Councillor Pete Smith said: "The fact of the matter is councillors nor the general public are aware of exactly what Walsall Council is storing and what its value is.
"There probably isn't the space to put everything on show but it then begs the question why have we got it if it is just gathering dust.
"If there is a reason to continue to possess these items that belong to the public, with Walsall Council custodians of public items, the maximum number of items should get the maximum amount of exposure." However leisure chief, Councillor Anthony Harris, said: "At nearly all museums and galleries, the treasures on display are but a small fraction of the riches within.
"With 2,500 metres of space, gallery room is limited and so many items in our collection are consigned to storage. Besides, it's good to change and refresh collections to avoid people thinking they have seen it all. Items on display are regularly rotated so the collection is always dynamic and changing. We also lend to other UK galleries and abroad."
Elsewhere, only 10 per cent of artwork owned by Wolverhampton council is on display to the general public, with thousands of pieces in storage.
The council owns approximately 17,000 pieces of art but 90 per cent remains locked away. The artwork in Walsall includes the Garman Ryan Collection collected by Kathleen Garman, later wife of the sculptor Jacob Epstein.