"All I know is that when I was young I joined the Air Cadets, and he told me to give it up," he says.
"He told me 'I got nothing out of it, and you will get nothing out of it', and that's all he really told me."
Like many who served with the British forces in the Far East, Fazal Hussain spent the remainder of his life haunted by the horrific conditions inside a Japanese prisoner-of-war camp. And Mr Qadus is calling for greater recognition of the role played by Muslim soldiers who served with the British Indian Army during the two world wars.
"You do hear about the Sikhs who served in the war, but you don't hear much about the Muslims," says Mr Qadus, 63.
"But if it wasn't for the contribution that the many Muslims made, it could very well be the Nazis or the Japanese ruling Britain today."
Mr Qadus points out that 1.5 million Muslims served with British forces during the conflict – not to mention the many more Muslims from the United States and Soviet Union who also took part – and is calling on the Government to create a memorial in recognition of their role.
Fazal Hussain was born in Rawalpindi in 1922, and joined the Army in his late teens, sometime around the outbreak of war. Mr Qadus knows little about his father's war record, other than that he was taken prisoner by the Japanese in Burma, and emerged from the prison as an emaciated shadow of his former self.
"I remember our grandmother saying that when he came back he was like a stick, and they had to carry him onto a hospital bed," he says. "He was really weak and took him a long time to recover in health wise."
Mr Hussain remained with the British Army until the partition of India in 1947, at which point he joined the newly formed Pakistani Army, rising to the rank of a non-commissioned officer. He retired from military service when he moved to the UK in 1962, where he settled in Lye and worked in the steel industry.
"He helped lots of the Asian community to get jobs," says Mr Qadus,who lives in Stourbridge. "He was a well known person in the Asian community for help and support."
He died during a visit to Pakistan in 2006, and Mr Qadus has been campaigning for greater recognition for the role of Muslim servicemen ever since, as well as compensation for those who were held prisoner by the Japanese.
Despite accounting for just a quarter of the population of British India, Muslims made up between a third and 40 per cent of the Indian Army during the Second World War, something that did not go unnoticed by Prime Minister Winston Churchill. He once told US President Franklin Roosevelt: "We must not on any account break with the Muslims, who represent...the main army elements on which we must rely for the immediate fighting."
It was a similar story in the First World War, when 400,000 Muslim soldiers served with the British Army.
Mr Qadus is not the only one calling for a memorial. Birmingham councillor Majid Mahmood is also calling for such a monument to be located in the city. He highlights the services of Khudadad Khan , who received the Victoria Cross, and Noor Inayat Khan, who earned the George Cross.
"A national memorial will remind the general public of the Muslim contribution on the side of the allies and the sacrifices made for all of our freedoms," he says.
Councillor Mahmood first called for a national memorial in the city in 2018, after the local authority gave the go-ahead for a monument in Cannon Hill Park to victims of the 2015 Tunisia terror attacks in Sousse and the Bardo Museum.
He said at the time: "I know there is some funding for a Sikh memorial in London but there is nothing for Muslims in the country and I think it is time we started a campaign.
"It is estimated 60,000 Muslims died whilst fighting for the British Army. People forget the roots of Muslims, where we have come from and where we have come to."
Councillor Mahmood points out that the Foreign and Commonwealth Office agreed to contribute to the cost of the memorial for the victims of the Tunisia attacks.
"None of the victims are from Birmingham but it has been chosen because the location is central," he says.
"But we have got people in Birmingham who have immediate family who were killed fighting for the British Army who were Muslim, but there is nothing in the country to recognise it.
"This is the ideal place for a memorial."
Khudadad Khan, a Sepoy of the 129th Duke of Connaught's Own Baluchis, became the first Indian soldier to be awarded the VC for his heroism during the battle of Ypres, in Belgium, on October 31, 1914.
The Baluchis were massively overwhelmed by the German attack but Khan, who was the only one left of his machine gun team, managed to halt their advance until reinforcements arrived.
Noor Inayat Khan volunteered with the Special Operations Executive and was the first female radio operator to be flown into occupied France during the Second World War.
Known by the name Norah Baker, her superiors initially dismissed her as 'too emotional' to work behind enemy lines, but she managed to send a large number of messages back to London while avoiding capture.
She was eventually arrested by the Gestapo, but refused to give up intelligence under interrogation, and was executed at Dachau concentration camp. She was posthumously awarded the George Cross, the highest honour available to a civilian.