Jean Markham, whose late husband's identity was stolen by Stonehouse, said he was "a scoundrel and a blackguard."
But to Stonehouse's daughter Julia, he was simply a caring, kindly man who suffered a breakdown due to the pressure of work and addiction to now-outlawed prescription medication.
Forty-five years ago, the Black Country MP and former cabinet minister was handed a seven-year jail sentence in what was then arguably the political scandal of the century. While it might have been eclipsed by the trial of Jeremy Thorpe three years later, the Stonehouse case was pretty heavy stuff. Not long before, the former postmaster general was being tipped as a future prime minister. Now he stood convicted on 18 charges of fraud, theft and deception.
But more than that, he had faked his own death only to re-emerge a month later having assumed the identity of a recently deceased constituent. If that were not enough, he was joined in the dock by his 29-year-old secretary Sheila Buckley, with whom he had been having an affair and allegedly plotting a new life.
Now, after decades of silence, his daughter Julia has finally decided to give her version of events in a new book.
Julia was 23 years old when her father went missing, and has up until this time kept her counsel about what life was like for her family during the bizarre events of the mid-1970s. So what has made her finally decide to spill the beans now, some four-and-a-half decades on?
"My brother and I have talked about doing something for some time, but my mother didn't want to reopen all this," she says.
"We've had nearly 50 years of people saying things that aren't true.
"But I'm 70 now, and you get to think 'when will you talk about it?' If I don't do it now, I never will."
While most people will remember Stonehouse as a suave, pin-striped politician and businessman who looked the epitome of the British establishment, Julia paints a picture of a doting father who shared her rebellious streak as a teenager. She tells a story about how, after being reprimanded at school for questioning why some pupils got more lunch than others, her father helped her in getting her revenge. One evening the two of them climbed over the school gate and left a rude message in the snow outside the headmaster's study.
Stonehouse, who had been Labour MP first for Wednesbury, and later Walsall North, was on business in Miami with his friend Jim Charlton when he disappeared on November 20, 1974. In the morning Mr Charlton had sat on the beach while the MP went for a swim, the pair later meeting with some banking executives. The MP said he wanted to go for another swim, and agreed to meet his friend in the bar at 7pm. He didn't show up. Mr Charlton went to look for his friend, and became alarmed when he realised Stonehouse's clothes were still in a cabana at the beach.
"Our first thought as a family was that my father had gone swimming far out from the shore, and had either had a bad case of cramp, a heart attack, or been eaten by sharks," recalls Julia.
She reveals that two days after his disappearance, her older sister Jane wrote in her diary how she hoped his body would never be discovered, so she could "imagine him starting a new life with amnesia."
As the days passed with the body still undiscovered, speculation grew about the missing MP. The Daily Mirror suggested he may have been kidnapped; her mother issued a denial to The Sun that money had gone missing from a fund to support the Bangladeshi government-in-exile, of which he was a trustee. On December 2, the BBC said the FBI were convinced he had not drowned, and there were suggestions he may have been murdered by the Mafia.
But the real bombshell came on December 9, when The Sun revealed that Stonehouse had secretly been renting a flat in Westminster, in which a 'slightly built brunette' had been living.
On being asked about this by a Sun reporter, Stonehouse's wife Barbara tried to contact his secretary Sheila Buckley, but for several days she was nowhere to be found. When Mrs Stonehouse finally did track her down, she asked her outright if she had been having an affair with her husband.
"Sheila was trying to behave normally, but from the look in her eyes she was terrified," says Julia.
"My mother asked her if she had been having an affair with my father, and Sheila broke down in sobs, with mascara running down her cheeks."
Mrs Stonehouse told Sheila that her husband had numerous affairs in the past, to which the secretary replied she thought she was pregnant with his child.
"What she hadn't told my mother was that she knew my father was alive, and that he knew she thought she was pregnant," says Julia. The reason that Mrs Stonehouse had been unable to find Mrs Buckley for several days was that she had gone on a secret trip to meet her lover in Copenhagen.
The secret didn't last long. On Christmas Eve 1974, the story took a sensational new twist when it emerged that Stonehouse had been arrested by police in Melbourne who thought they had discovered Lord Lucan.
Lucan had vanished 12 days before Stonehouse, and he was one of the most wanted men in the world after apparently murdering his children's nanny. Suspicions were raised when "a theatrical sort of fellow in tweeds" with a plummy English accent started transferring large amounts of money using the names Joseph Markham and Donald Muldoon. Police thought they had found the fugitive peer.
Wife Barbara told the Express & Star the news was "the best Christmas present we could ever have had", but the joy would be short-lived.
When Barbara visited him in Melbourne, her husband took a call from his secretary and she overheard them arranging a meeting in Perth. Julia says when her mother threatened to return home if he met Sheila, her father became violent for the first time in his life.
"My father lost control. He grabbed my mother and threw her to the floor, yelling 'why can't you understand?'
"My mother was face-down on the floor and my father leant down, grabbed her hair, and used it to bang her head up and down on the floor." Julia's 14-year-old brother Mathew pulled Stonehouse off his mother and told her to go to the kitchen.
"Usually, my father was so gentle," says Julia. "He could be emotionally cruel, but never violent. In the bedroom, he was banging his head against the wall and crying his heart out."
Barbara attempted to call her husband's psychiatrist, but Stonehouse assumed she was calling the police and pulled the phone from the socket, attacking her a second time before threatening to kill himself and speeding away in his car.
Julia suggests her father's outlandish behaviour may have been down to withdrawal symptoms from the sleeping tablets Mandrax which he had become increasingly dependent on.
"My father's manic behaviour was so out of character it was frightening," she says.
Stonehouse was convicted of 18 charges of theft or deception, Buckley five. He was handed jail sentences totalling more than 95 years, although most of these were concurrent, meaning that in reality his term would be seven years. Buckley was given a two-year suspended sentence.
Julia acknowledges that laws may have been broken, but says the sentence was disproportionate.
"Should a man go to jail for having a breakdown?" she says.
She says that many of the theft charges were simply technical breaches, taking money from companies that he owned.
"Yes, there were irregularities, but not exactly criminal."
She adds that five of the charges related to taking out £125,000 worth of life insurance in the event of his death.
"There was a seven-year time limit on those policies, and they would only pay out in the event of a body being found," says Julia.
"The only reason he took those out was because six months earlier his car was blown up by an IRA bomb."
Julia also insists that Buckley was innocent of the charges, convicted only on the basis of a dubious newspaper report which had mistaken a trunk of her mother's clothes for those of his secretary.
"People think she was in on this conspiracy to start a new life in Australia, but she knew nothing about it," she says.
While in prison, Stonehouse suffered three heart attacks, and was released in August 1979 having served three years. On his release he married Sheila, and began a new career as a novelist. He was taken ill during an appearance on the notoriously fiery late-night debating show Central Weekend in March, 1986, and died three weeks later after suffering another heart attack. His first wife, Julia's mother Barbara, is now aged 90.
While his life arguably took more twists and turns than any of his book characters, Julia has remained steadfastly loyal to her disgraced father. Given that the Stonehouse family has probably suffered more than anyone else from his crimes, does that mean she has an unusually forgiven nature?
"As far as I am concerned, there is nothing to forgive him for," she says.
"My mother had an issue over the affair with Sheila, but that's a separate issue, it's none of my business, I don't judge him for that. People have affairs."
She says she hopes her book will set the record straight about some of the more lurid claims that have been made about her father.
"People have said that he stole from a charity, that he was a Communist spy," she says.
"He was very anti-communist, I remember stuffing envelopes for him while he was trying to stop the London Co-operative Society from being taken over by communists."
Her book dissects claims, which emanated in the late 1960s, that her father was working for the Czech secret service. She has obtained a copy of his Czech security file, which she says not only has an incorrect address for where they lived, but also contains documents supposedly written by her father, but which bear no resemblance to his own handwriting.
She accepts that trying to rehabilitate her father's memory is a difficult task after decades of negative press.
"People are going to believe what they have been told for nearly 50 years," she says
"My hope is that in 20 or 30 years' time, when all our generation has died and gone, that some young kid will check out this Stonehouse girl and what she said."
John Stonehouse, My Father – The True Story of the Runaway MP, by Julia Stonehouse, is on sale now.