The retirement units will replace a tired old office building on the High street which is set to be demolished.
Developers, Churchill Retirement Living & The Truste, say the retirement units will help to deal with the swelling elderly population in the borough.
But residents living close to the proposed scheme say the lack of privacy and noise from construction would make their lives very difficult.
One resident said: “Our house backs onto Hampshire House and our garden is to be the boundary wall of where the applicant wishes to build. Currently, we have nobody overlooking our property and no noise.
“We are concerned about a lack of privacy and people being able to look into our house and garden. We also work various shifts including night shifts and the building work will be unbearable.
“We will lose our view at the back of the house which will be replaced with a concrete building. There will be a huge loss to our property and a potentially huge loss in the value of our property too.”
The new retirement community would see 47 living units installed with a reception desk, owner’s lounge coffee bar and other communal facilities. The apartments would be a mix of one and two beds as well as three cottages and space for 17 cars.
There are currently 69,300 people over the age of 60 within the current population of Dudley and Churchill Retirement says the retirement scheme could generate £550,000 of spending a year for the local community.
They also say a public consultation delivered to 307 residential and business properties surrounding the site only resulted in two letters of feedback.
In a design and access statement, a representative for the developer said: “Churchill Retirement Living’s vision for the site is to deliver a development that meets our customer’s needs and the local need for retirement apartments whilst contributing to the character of Kingswinford and making a positive contribution locally in terms of social, economic and environmental benefits.
“Our aim is to create a high-quality development that embraces sustainable design, enhances the setting of the area and maintains the local vernacular. The proposals make the best use of this highly sustainable site in close proximity to the town centre and provide retirement living accommodation in line with the recognised local need.”