Some 19 inmates have now tested positive for coronavirus in 10 different prisons.
The number of cases, which the Ministry of Justice (MoJ) said was accurate as of 1pm on Wednesday, has jumped up by six in 24 hours.
Jails in England and Wales were put on immediate lockdown on Tuesday with all visits cancelled as it emerged thousands of staff were in self-isolation.
One of the confirmed cases is at HMP Oakwood, Britain's largest jail which is just north of Wolverhampton.
Jo Farrar, chief executive of the Prison and Probation Service, told the Commons Justice Committee around 4,300 prison and probation staff overall were self-isolating at present.
Some 3,500 were prison staff – representing about 10% of the workforce – and four have tested positive for Covid-19, each in separate jails.
There have been three prisoner escort and custody services staff and eight probation staff who have also tested positive.
Around 50,000 protective masks have been delivered for staff to use and a ban on bringing hand sanitiser into the prison had been lifted.
Justice Secretary Robert Buckland has hinted he may have to consider some forms of early release for prisoners in a bid to ease pressure on jails during the crisis.
Peter Dawson, director of the Prison Reform Trust, said: “The risks of infection for people who live and work in prisons are acute, as the Justice Secretary has said.
“So while it is obviously right that the way of life in prisons should change dramatically, just as it has in the wider community, Robert Buckland is right to be considering forms of early release.
“In a chronically overcrowded system, this will help keep people safe by reducing the pressure on both space and staff resources.
“For this to work, there will need to be support to organisations, many of them charities, that help prisoners on release.
“And it’s vital that while creating some headroom through releases, the flow of people into prisons is also drastically reduced.
“That means not sending anyone to prison for all but the most serious alleged or proved offending, and not recalling people to prison in all but the most dangerous of circumstances.”