He was reported to be in the water near The New Navigation off Titford Road, in Oldbury, early on Wednesday morning.
Police, paramedics and firefighters were sent to the scene and pulled him from the canal but he could not be saved.
The death of the man is not being treated as suspicious.
A spokesman for West Midlands Police said on Wednesday: "We received a 999 call reporting a body in the canal off Titford Road, Oldbury, at around 6.40am.
"A man was rescued from the canal but he was sadly pronounced dead at the scene.
"We’re working hard to identify him so that we can pass on the sad news to his family.
"At this stage his death is not being treated as suspicious and the coroner has been informed.
"The charity Samaritans is available 24/7 should you need to talk."
A spokesman for West Midlands Ambulance Service added: "“We were called by the police at 6.46am on Wednesday to reports of a patient in the canal near the Navigation Inn, Titford Road, Oldbury.
"Two ambulances, a paramedic officer, a critical care paramedic and HART paramedics were sent to the scene.
"On arrival crews found a man in the canal.
"Crews worked quickly with fire colleagues to rescue him from the water before ambulance staff commencing advanced life support on scene.
"Sadly, despite their best efforts nothing could be done to save the man and he was confirmed dead a short time later."
If you have been affected by this article, you can call the Samaritans on 116123 or visit Samaritans.org