The former Willenhall Town Gasworks site, now known as the Stonegate Housing Estate, was identified as contaminated by Walsall Council back in 2007.
After 10 years of back and forth with the company, the authority was deemed responsible for the clear-up.
However, residents have spoken of their concern that this may take years itself.
Jo Comley, 66, has lived on Brookthorpe Drive for 25 years and said: "Life has essentially been put on hold for 10 years while this has been going on and it hasn't been fair on us.
"We have been stuck in the middle, absolutely incapable of doing anything to move the situation along.
"I know of people that have wanted to start a family and have decided to put that on hold, people who want to extend their home or do things to their property and they just haven't been able to.
"Everyone has their concerns and I hope it is going to get sorted as soon as possible now."
Julie Parsons, 60, who has lived on the same street for 30 years, said: "It has just been like walking into a brick wall for 10 years.
"Everyone has been affected in a different way, but for us it has been the fact we haven't been able to move on and look at extending the house or anything like that.
"You still worry that even when it is cleared up, there will be a stigma surrounding the estate and maybe our house prices won't recover. "
Violet Lea, 65, who has lived on Kemble Close for 40 years, said: "It has been a nightmare really, we have never had a proper conversation with anyone because they were always shirking responsibility.
"We have needed to build an extension to our home for a long time but we just can't do it. The value of our properties is next to nothing when you compare it with houses that are just a few streets away.
"It is a lovely neighbourhood but this has just hung over us like a stigma for so many years, it is always in the back of your mind. "
Liz Bailey, 46, moved to the estate in 1999. She said: "I have a growing family and I have needed to build an extension for years but we can't do it. It really has been a nightmare, we haven't had the right communication from anyone about this.
"We are left in the middle of it all, having to live with the effects and not knowing where we stand.
"People are worried about leaving their houses to their children because of the trouble they will have selling it.
"I don't know how much faith I have that the council will get this done, but everybody wants and needs this to be sorted out as soon as possible."
Bernard Richards, 61, has lived on Kemble Close for 32 years. He said: "It has taken such a long time to get to this point but at least now we know who we can take our issues up with. Lets hope that this job is done right and in good time."
A remediation notice was served on Jim 2 Limited, the firm the authority claims was liable for the contamination, in 2015.
But the company lodged an appeal against the clean-up notice with the Planning Inspectorate, which led to a public inquiry.
The inquiry findings went to the Secretary of State for environment, Dr Therese Coffey, who has now ruled that the notice should not have been served, leaving Walsall Council responsible for the clean-up work.
Following the news, council leader Sean Coughlan said the authority was 'disappointed' with the outcome but would be looking to take the matter forward 'in the interests of residents'.