Officers were called to Dudley Road East, Tividale, just before 1.30am on Sunday, where the victim was found with serious gunshot wounds.
He was treated at the scene by paramedics before being taken to hospital but was later pronounced dead.
The street was cordoned off and a forensics tent was put up at the scene which was later opened up to reveal a blue taxi inside from 121 Cars, based in Tipton.
The victim is believed to have been a passenger in the taxi. The driver escaped without injury but suffered severe shock. The firm did not wish to comment but it is understood the driver was helping West Midlands Police with enquiries.
A West Midland Police spokesperson said: "Our investigation is in the early stages and we have closed off the area as forensic examinations are carried out."
A section of Dudley Road East was cordoned off on Sunday morning as forensic investigations got underway.
A blue police tent was located upon the road, on the left hand side of the carriageway, heading in the direction of Oldbury.
Inside the tent could be seen a blue taxi from the firm 121 Cars.
The cordon stretched from between two businesses located on the street, although the tent itself was located further inside, next to some bushes and residential houses.
Residents living near to the scene spoke of their shock, saying crime was not usually an issue in the area.
Jaz Kaur, aged 57, said she heard an ambulance during the early hours, adding: "I thought someone had crashed.
"Yes, of course the incident is shocking. We have never heard anything like this before, we have never seen anything like this.
"You wouldn't expect it along a main road with traffic passing by."
Lynne Austin, 43, said: "I am shocked to hear about this. This does not really happen around here, it is a pretty quiet area.
"I am just wondering what has happened and who it could be."
One man, who wished to remain anonymous, said heard a bang and thought it was a gunshot.
He said: "Jokingly, I said to my wife I heard a gunshot. I was sitting down, watching the TV when I saw the police. I thought there had been a car crash.
"I am shocked to hear about this, very much so. I have lived here for 18 years, I am really surprised. Crime is not an issue around here."
Anyone with any information is asked to contact police via live chat at west-midlands.police.uk or call us on 101, quoting crime number 20/1126386/21.
Alternatively, contact Crimestoppers anonymously on 0800 555111 or at crimestoppers-uk.org