The discovery was made by a member of the public at around 2.20pm yesterday near the junction between Lewisham Road and Bridge Street North in Smethwick.
The body, believed to be that of an elderly man, was found in a narrow section of the Birmingham Canal near to a lock, underneath the Rolfe Bridge.
He has not yet been identified, and his age is also unknown at this stage.
It was found in an industrial area of the town with several factory and warehouse units nearby, opposite the now derelict New Navigation public house.
That section of the canal, which is next to a small car park, was yesterday afternoon sealed off by police and a blue tent was erected next to the waterway as they investigated the scene.
Around 10 officers could be seen working next to the tent, while others could also be seen in the water.
An ambulance and four police vehicles were in attendance.
Traffic travelling over the bridge was slowing as drivers and passengers looked to see what was happening.
A the time of the find, West Midlands Police spokesman Lee Page said: "We can confirm a man's body has been found in a canal. We were called at 2.20pm to Bridge Street North at Lewisham Road, Smethwick. The area has been cordoned off."
But today police have said the incident is not believed to be suspicious, and the case has now been handed over to the coroner.
In May 2012 the body of 56-year-old Balbeer Singh Srah was also found in the canal by a passer-by, near to the Spon Lane bridge in Smethwick.
An inquest into his death in July hears that despite an extensive police investigation and a post-mortem, there was insufficient evidence as to how he died.