Michael Davies had helped to remove roof material containing the substance in 1978.
The 60-year-old, of Green Close, Pattingham, died on September 18 this year. He was a retired social care manager.
An inquest this week heard that the cause of his death was mesothelioma – a type of cancer that develops in the lining that covers the outer surface of organs, and is usually linked to asbestos exposure.
A statement from Mr Davies’ family was read out at Cannock Coroners Court by officer Andrew Heathcote.
He said: “After graduating in 1978 in Swansea Mr Davies had no permanent employment and it appears he survived by doing odd jobs.
“He remembered his friend Tony returning from the Job Centre to say there was a job clearing asbestos in a factory in Swansea.
“He thought the opportunity was too good to turn down and signed up. But after just two days dismantling roofs, he gave it up.”
Mr Davies reported this exposure to asbestos in July this year.
However, he told the court that the period between exposure to asbestos and diagnosis of the condition, can be anything from 20 to 60 years.
Mr Heathcote continued: “The initial exposure happens decades before symptoms start to show.”
South Staffordshire Coroner, Mr Andrew Haigh, said: “I have been told Michael was only 60 when he died, which is a relatively young age to die.
“He was an active man, had a responsible job in social services and was part of the community in Pattingham.
“After he had obtained his degree in Swansea in the late-1970s, he had a job for a couple of days removing asbestos. He reported this to the chest physician, who said they would expect there to be longer exposure than just a couple of days.
“There is no doubt that Mr Davies died as a result of mesothelioma and no doubt that this was a result from exposure to asbestos.
“What is more difficult is when the exposure occurred, and the time that he was working in Swansea is the only exposure I have been told about.
“Therefore, on the balance of probabilities, I conclude Mr Davies died of industrial disease.”