Dudley Muslim Association lodged the appeal after its defence to Dudley Council's action to reclaim land earmarked for the multi-million pound development in Hall Street was struck out at an earlier hearing.
But Deputy Judge David Halpern QC upheld the earlier ruling. He said while the association may have been able to resist the council's claim, it had failed to serve a 'credible' defence during the proceedings.
And he said it was now too late for the organisation to put its case in order at the hearing on Friday.
The row goes back to 2003 when the association agreed to give the council a site it owned in Claughton Street, needed as part of a development, in return for a long lease over the disputed plot between Hall Street and Oakwell Street.
It agreed to begin building work within five years or the lease would come to an end. But when the scheme came before planners in March 2007 it was rejected.
A planning inspector overturned the decision and the council failed in a legal decision against that ruling.
But by then the deadline for the lease had passed and the council launched the proceedings to reclaim the land in 2010.
The judge said the council viewed the land as a high-profile site that it did not wish to remain derelict.
Dudley Council leader Councillor David Sparks said the authority's lawyers would now be looking at the judgment before giving advice on how to proceed. "This is a very complex legal situation and as leader of the council I have been very careful to act upon the advice of the council's lawyers," he said.
"They will now be studying the judgment fully before advising us on our next course of action." The Dudley Muslim Association was unavailable for comment.