It is understood he went into Middle Pool at Fens Pool Nature Reserve to retrieve a ball at around 6.15pm today and then did not resurface.
Resuscitation attempts were then made before he was taken to hospital.
But this evening, West Midlands Police confirmed the man had later died.
Dominic Rowley, aged 31 from Brierley Hill, saw the drama unfold.
He said: "We were sitting on the bank and saw a man and woman with one or two kids. A football went in the water and he started walking in after it and started taking his shoes and socks off.
"There was a strong wind. The ball was only five or six feet out but when he got in the ball went further and further out and within 10 seconds he went under.
"He never resurfaced. I phoned the ambulance. Police turned up first and then paramedics.
"They waded in with long poles and life jackets but he was under the water for at least 45 minutes before they got him back out."
Police want anyone who saw what happened to call them on 101.