The council had written to residents outlining proposals to carry out "ground investigation works" on Goldicroft Park.
The letter, published last month, said the work would "inform the suitability for the redevelopment of the site for housing".
Sandwell Council leader Rajbir Singh said in a statement on Wednesday: "The information from the site investigations would have been used to develop options for the future of the site, which would have been subject to further public consultation before any decision was taken.
"However, given the level of objection to the site investigations, and listening to the voices of ward members and residents I have agreed, as part of my commitment to protect Sandwell’s green spaces and build a cleaner, greener, safer borough, that we will not be progressing with plans for new housing on Goldicroft Park."
Welcoming the decision, Wednesbury North councillor Luke Giles said: "I am delighted with the news and it is the right decision. The community came together on this. I had a number of conversations with senior members of staff."