A probe into Chase Nursing Home also revealed a lack of activities on offer and staff were accused of using residents' bank cards to withdraw cash on their behalf with no evidence of consent.
The Care Quality Commission (CQC) said it could not be sure people were protected from the "risk of abuse" and also criticised poorly-kept records.
The report said: "We could not be certain that people experienced care and support that met their needs and protected their rights."
It revealed Walsall Council had received four allegations of bullying by staff and new admissions had been suspended since May. It also claimed there were not enough experienced workers at the Vicarage Road home.
According to the report, an acting manager was in post at the time of the June 27 inspection but he had not previously managed a care home and no formal reference had been given from a previous employer.
It said residents lacked "meaningful activities" and one person was observed sitting alone outside "smoking one cigarette after another for an hour".
Warning notices were issued to owner Heritage Healthcare (Midlands) Ltd calling for an action plan to be drawn up. Enforcement action could follow if things do not improve.
Andrea Gordon of CQC, said: "The law says these are the standards that everyone should be able to expect. Providers have a duty to ensure they are compliant."
She said inspectors would return in thefuture to see if progress had been made.
A member of staff at Chase Nursing Home, who would not be named, said: "All areas identified have been dealt with and the CQC has been informed."