Paramedics were called to Brasshouse Lane at about 7.50am on Monday and found a man in a critical condition.
He was confirmed dead at the scene and his death is not being treated as suspicious, West Midlands Police said.
Jamie Arrowsmith, spokesman for West Midlands Ambulance Service, said: "We were called to reports of a medical emergency on Brasshouse Lane, Smethwick at 7.51am.
"Two ambulances, a paramedic officer, the Midlands Air Ambulance Critical Care Car and our Hazardous Area Response Team attended the scene.
"On arrival we discovered one male patient who was in a critical condition.
"Ambulance staff immediately began administering advanced life support but sadly it became apparent nothing could be done to save him and he was confirmed dead at the scene."
A spokesman for West Midlands Police said: "We were called after a man in his 70s was found unresponsive on a canal towpath near Galton Bridge, Smethwick, at around 7.50am today.
"Nothing could be done to revive the man and he was sadly confirmed dead at the scene.
"The death is not being treated as suspicious and a report will be prepared for the coroner."