Inspectors from Care Quality Commission (CQC) revealed the findings after examining Walsall Manor Hospital in September this year.
They examined the hospital's maternity services and urgent and emergency care department at the hospital amid the coronavirus pandemic.
They rated the hospital as Requires Improvement overall – the same as last year – with the rating for being "well-led" in the departments falling over concerns.
Inspectors found the hospital didn't have "enough" nursing and support staff in the emergency department to keep patients safe from "avoidable harm", the CQC report showed.
And documentation of sepsis screening and management needed to be more robust in the department and needed further scrutiny to ensure the safety of patients.
The rating for being well-led fell by one – from Good to Requires Improvement – in the Urgent and Emergency Services category due to leaders and teams not always managing performance effectively.
Inspectors found they didn't always identify and escalate relevant risks or issues, but they did have plans in place to cope with unexpected events – with waiting times being in or better than national standards.
They found the service had enough staff – with the right mix of qualifications and skills – to keep patients safe in the maternity department, the CQC report showed.
But concerns were raised over systems for monitoring the provision of staffing which were found to be "not robust" by inspectors – leading to the well-led category in the maternity department dropping from Good to Requires Improvement.
Inspectors noted in that staff in the emergency department felt "respected, supported and valued" and the service had "an open culture" – where patients, families and staff could raise concerns without fear.
And staff also felt "positive and proud" to work in maternity services and supported to develop in their roles with leaders appreciating the day to day pressures they faced, the report showed.
Walsall Healthcare NHS Trust’s chief executive Richard Beeken said: "This year has seen an unparalleled demand on the services we provide as a result of the global Covid-19 pandemic.
"The response of our staff, along with key workers across the country, has been nothing sort of phenomenal, in circumstances that go far beyond “challenging.”
"The pace has been relentless and the impact that the pandemic has made, and continues to make, has taken its toll in many areas - this includes staffing.
"That said, we are disappointed to have gone down a rating in well-led. We have made it clear that our ambition is to become a trust that is rated outstanding by 2022 and that focus has not wavered, despite these testing times.
"We know we still have a long way to go to ensure that our services are the best and safest they can be for all the communities we serve and our commitment to achieving this remains."