Councillors approved plans to use the four-bedroom building on Gorway Road as a residential care home for three children at a meeting on Thursday, which will see young people of any gender aged between eight and 18 looked after at the house.
Two residents spoke at the meeting against the plan, and a petition in protest with more than 50 signatures on was also handed into the council.
One of the residents said: "There is going to be more crime in the area, the other residential homes nearby are older people so they are quite docile, there is no noise from them."
Another objector added: "I live next to the property in question, we're concerned that the proposal will affect our daily lives, we moved here to retire."
But another speaker, on behalf of the applicant Sanah Akram, said: "It will be a care home for looked-after children who have suffered trauma and neglect, and it will provide a suitable and safe home.
"All children deserve to live in a safe environment, imagine these were your children.
"Our days are spent building positive relationships just how we would with our own children.
"Every day look-after children and care leavers face discrimination a well as having to fight stereotypes.
"As a community we should come together and help to support these young people."
Councillor Steve Craddock said the facility is required in Walsall.
He said: "It's a sad reflection that we actually need this facility in the area but they are required.
"I'm encouraged that the level of care is one-on-one in the day and two-to-one in the evenings and I think we need to approve it."
Councillors voted in line with officer's recommendations and approved the application.