Bearwood-based Highgate Brewery Tap House had also hoped to put on plays, films, live music and dances at the Victorian building in Sandymount Road.
Following objections from people living nearby, the company said it would be "pointless" going forward with the plan if it was unacceptable to the people it was meant to serve,
The company has issued a statement saying it had always had the best interests of the community it serves at the heart of their business.
"For many years, the brewery employed local people and also enjoyed support from the Friends of Highgate Brewery," it said.
"Since submitting the application for the tap house, they have become aware of mounting objections and concerns by local residents with regards to the proposed plans. Having listened to those objections, the board has decided to withdraw their application with immediate effect as they will no longer be able to run a viable business from the site.
"It would be pointless progressing with a plan that clearly is unacceptable to the people that Highgate Brewery had hoped would embrace, and enjoy the tap house. Highgate Brewery will make no further statement with regards to the withdrawal of the application."
Residents living nearby were concerned that opening a bar on the site would cause disturbance and traffic problems in a primarily residential area that was also a conservation area.
An application to get an alcohol and entertainment licence for the Grade II listed brewery had been submitted to Walsall Council.
The deadline for objections was on Tuesday.
The brewery was built between 1895 and 1898, and closed in 2010. It was famous for the brew Highgate Dark Mild, and at its peak employed 65 workers – producing 100,000 barrels a year and providing its beer to more than 400 pubs and clubs across the region.
The Friends of Highgate Brewery – set up to help preserve the building – and Walsall branch of the Campaign for Real Ale had supported the plans to get the site back in operation.
The five-storey brewery has failed to sell at several auctions in recent years.