Andrew Wood, from Hereford, and Mark Shelton, from Shifnal, allege former Tettenhall College headmaster Colin Cope sexually abused them as pupils in the 1970s. Cope died aged 84 on Thursday in Dorset.
He was under fresh investigation by police after secretly-recorded video footage emerged showing the ex-teacher driving his car, carrying a table and climbing steep stairs while giving paying visitors a guided tour of his 18th-century National Trust home.
This was just eight months after the court case against him, which was being held at Shrewsbury Crown Court in 2009, was halted by Judge Robin Onions because he deemed that the defendant was 'too ill' to continue with the proceedings.
Cope was charged with 11 offences on five boys and denied the offences.
Mr Wood and Mr Shelton were among five ex-Tettenhall College pupils who were paid a total of £129,000 in a civil claim by the boarding school in an out-of-court settlement in 2012.
Mr Shelton, 57, said: "This is a classic case of justice delayed being justice denied. It is difficult to sum up how I feel. Even though he has died, I feel strongly that there are unanswered questions. I won't rest until we have those answers."
Mr Wood, 56, added: "A opportunity has been missed for justice to be done. Questions have to be asked of the handling of the case over the last nine years by the police and the Crown Prosecution Service. We have been denied the opportunity for Colin Cope to answer the charges brought against him in a court of law."
The alleged victims gave statements to police claiming the sexual abuse took place in Mr Cope's private rooms at the school, as well as on camping trips, in a boy's bedroom and in his car.
Cope taught at the independent school between 1959 and 1973 where he became headmaster of the lower school. He lived in a National Trust property in Sturminster Marshall, Dorset and died on Thursday, his solicitor Julian Hardy confirmed.
He was first arrested in 2006 and police interview transcripts seen by the Express & Star state that he told officers "I just know that I have done things I shouldn't have done in the past." In another interview he describes to police how he moved to Wentworth Manor Girls School in Bournemouth as 'jumping out of the frying pan and into the fire.'
When the 2009 court case collapsed, Cope was said to be taking 12 medications for numerous illnesses and had heart disease, prostate cancer and diabetes.
The Express & Star revealed in November how police had started reviewing the footage and later passed it on to the Crown Prosecution Service in January this year.
The pair had been supported in the pursuit for justice by Wolverhampton South West MP Paul Uppal who arranged a meeting with the Home Secretary Theresa May last month. West Bromwich East MP Tom Watson has also championed their case.
Mr Uppal said: "There has to be a posthumous investigation into this. For the survivors they will feel that they have been denied closure. I have already written to the Chief Constable of West Midlands Police over this case and while Colin Cope's death closes one door, I think it opens another.
"It is even more pertinent now that we get to the truth." The Home Secretary has requested the two men register to give evidence to the national child sex exploitation inquiry headed by New Zealand judge Justice Lowell Goddard. The inquiry will be considering whether, and the extent to which, public bodies and other important institutions, including boarding schools, have taken seriously their duty of care to protect children from sexual abuse.
It will seek to address public concern over failings exposed by appalling cases of organised and persistent child sexual abuse and cover cases back to 1945.
The CPS said the review would continue until a death certificate was produced.
West Midlands Police were yet to respond to inquiries by the Express & Star at the time of print but senior officers are understood to be aware of Cope's death.
Tettenhall College declined to comment.