Council worker Jon Beesley claimed The Royal Wolverhampton NHS Trust was to blame for the 'worsening condition' of the crumbling site since its closure more than a decade ago.
It comes after the trust alleged an enforcement notice demanding it overhauls the Compton Road site was served 'abruptly' by Wolverhampton Council.
Ms Alison Ogley, on behalf of the trust, put it to Mr Beesley that the authority does not have a statutory duty to 'protect, preserve or maintain' any of its buildings.
Mr Beesley told Dudley Magistrates' Court: "I'm not suggesting that the trust has a statutory responsibility, rather, a moral responsibility, as a responsible public body, responsible for buildings in its ownership.
Ms Ogley then put it to the witness: "The trust can't have failed to have done something that it isn't required to do?"
More on the eye infirmary:
Mr Beesley, the council's then urban design and conservation officer, replied: "Yes, [but] there has been a lack of maintenance on the site which resulted in the deteriorating condition of the buildings."
The two authorities are currently locked in a courtroom battle after the council served the trust with the notice, ordering it to sort a raft of issues within seven months.
Mr Beesley told the hearing the impact to the surrounding area was the 'trigger' for issuing the notice last February.
But Ms Ogley claimed the council 'took no action' to resolve issues despite a 2015 report revealing the buildings were in a 'serious state of disrepair'.
Mr Beesley said on Friday: "It would be incorrect to say that the council was ignoring the site. It was actually working with the trust to find a solution.
"I wouldn't say we were aware of the serious structural problems. We have had no structural engineer's report."
The council served its enforcement notice after claiming the trust failed to carry out required repair work.
It has since said not all work was completed by the final September 20 deadline and could now seek prosecution.
But the NHS trust, which is calling for the notice to be ‘quashed’ or amended, lodged an appeal.
The eye infirmary, which opened in 1888, closed when services moved to New Cross Hospital and has stood empty since 2007.
The hearing continues.