The man, who is thought to be in his 20s, was stabbed a number of times in the chest and back at around 2.30pm on Friday in Summerfield Park, Dudley Road, Winson Green.
He was taken to hospital for treatment, but was pronounced dead just after midnight on Saturday.
Police believe they know who the victim is, but are working to confirm his identity and trace his next of kin.
The force received reports that gunshots were heard, and officers remained at the scene as forensic work continued.
A glass panel in a door in the area has been smashed with a blast style of hole, officers said.
A post-mortem on the victim will be held in due course.
Detective Inspector Laura Harrison, leading the investigation, said: "This was appalling violence which has ended in a man's murder, and we've been working non-stop since yesterday to establish exactly what happened, and who was responsible.
"I really need to hear from anyone in the area at the time who saw or heard anything and who's not yet spoken to us.
"We also really need to speak to anyone with CCTV, mobile phone or dashcam footage that might help us."
The air ambulance was scrambled and landed in Summerfield Park. West Midlands Ambulance Service said: "We sent the Midlands Air Ambulance from Cosford, two land ambulances and two paramedic officers.
"We treated a man for serious injuries and he has been taken by land ambulance to hospital."
Anyone with information been asked to get in touch by calling 101, quoting log number 2416 of 4 December.
Alternatively, contact Crimestoppers anonymously on 0800 555111 or at crimestoppers-uk.org