Two families including a baby were evacuated from their flats when cracks opened up in the walls and kitchen surfaces in their homes began to move.
Firefighters were called to the three-storey building in High Street, Lye, by a concerned resident living in one of two flats emptied last night.
They discovered the building was unstable due to work being carried out to knock down a wall between two ground floor shops and took the decision to evacuate the properties above.
The eight residents were looked after in a mobile welfare vehicle by the fire and emergency rescue service before being taken to alternative accommodation for the night.
The fire service’s technical rescue team gained access to the property, next to the Garam Masala restaurant, and used wood to temporarily secure the building.
They were returning today, along with structural engineers from Dudley Council, to assess the damage.
Gav Pardoe, manager at Stourbridge fire station, said: "The worst case scenario would have been the wall coming down and the building collapse with people inside."
He added: "It had the potential to be very serious - building work was going on in the ground floor and the builders were a bit too keen in removing some of the brickwork before making sure the first floor was stable.
"Cracks began to appear in the wall which goes up between two flats above, the kitchen tops also started to move. It was not safe inside."
Two families of four live in each of the two flats above the shops.
Mr Pardoe said: "They are now being looked after by our teams.
"They were quite shocked - they didn't realise the seriousness of the situation until we told them."