The former Willenhall Town Gasworks site, now known as the Stonegate Housing Estate, was identified as contaminated by Walsall Council in 2007.
A remediation notice was served on Jim 2 Limited, the firm the authority claims was liable for the contamination, in 2015.
But the company lodged an appeal against the clean-up notice with the Planning Inspectorate, which led to a public inquiry.
The inquiry findings went to the Secretary of State for environment, Dr Therese Coffey, who has now ruled that the notice should not have been served, leaving Walsall Council responsible for the clean-up work.
A letter signed by Mrs Coffey reads: "I have determined, in line with the inspector's recommendations contained in the report, that the appeal be allowed and the remediation notice quashed."
MP David Winnick said residents had 'been through hell' and urged the council to start work as soon as possible. He said: " I think hell is an appropriate term for what they have had to endure. No-one has admitted responsibility for the contamination and it is the residents who have had to deal with the fall out from that. I hope that Walsall Council accepts this decision and moves forward soon."
Almost 90 homes on the site of the former gas works near Kemble Close, Oakridge Drive and Brookthorpe Drive in Willenhall were affected by the contamination.
Residents previously said they were worried about house prices falling and growing vegetables in their gardens after it emerged their gardens were polluted.
It has previously been claimed that the clear-up could cost around £2.5 million.
Council leader Sean Coughlan said the authority was 'disappointed' with the outcome but would be looking to take the matter forward 'in the interests of local residents'.
"The council is of course disappointed that the remediation notice was quashed on such a hugely complex, technical matter and is now considering how best to take this matter forward."
Jim 2 Limited is listed as a dormant company and was unavailable for comment.