Mr Fabricant said thousands of staff would be endangered due by the move – along with MPs – due to the risk of infection.
And he said one of his colleagues had already described the House of Commons as a "morgue".
It comes after Commons leader Jacob Rees-Mogg said virtual proceedings were stifling scrutiny and debate – and instructed MPs to return today.
Mr Fabricant, who will carry out his duties virtually, said: "A colleague of mine said the place is like a morgue and the Speaker has said 27 Conservative MPs will be allowed in the chamber at any one time.
More Covid-19 coverage:
"None of the facilities will be open and the risk of contagion in the narrow corridors, dating back to the 1840s, is immensely high – endangering the thousands of staff who work at the House of Commons, let alone the MPs.
"Moreover, it's an epidemiologist's nightmare having people travel from parts of the country where there's a high rate of infection to London which is now currently past its peak."
Mr Fabricant previously said the virtual arrangements were far from perfect – but allowed all MPs to carry out their work safely.
Other MPs said they looked forward to returning to House of Commons, saying it had been tough trying to debate virtually.
Stuart Anderson, who had suffered from coronavirus symptoms, said he was looking forward to returning – but said he had enjoyed his time with his family.
The Wolverhampton South West MP said: "It's going to be very strange and as we've been told there will only be about 50 MPs in the chamber.
"It's hard to deal with Westminster in Wolverhampton – it's easier to look after the city in Westminster.
"It's hard to debate and it's hard to stay updated because you have to watch a screen, so I'm looking forward to getting there.
"I'm very uncertain about how it's going to work but we do need Parliament to sit – there has to be scrutiny on all the bills and there's a lot we're trying to do as a Government."
Eddie Hughes, MP for Walsall North, said the virtual House of Commons worked for a short period of time – but they needed to get back into the chamber.
He said: "It's kind of tough, but I think it worked for a short period of time – but now the rest of the country is coming out of lockdown.
"People are going to their jobs and children are returning to school and I think it's important MPs are seen to follow that process."