Eighteen babies have been struck down with blood poisoning after contracting an infection from a suspected contaminated drip.
The cases of septicaemia, which were bought on by an infection that has been "strongly linked" to an intravenous fluid, has killed one youngster and left others needing treatment with antibiotics.
The babies were being treated in neonatal units at nine hospitals around England, Public Health England (PHE) said.
They had all been given a fluid called parenteral nurition - which is supposed to deliver a variety of nutrients intravenously when a baby is unable to eat on its own.
Health Secretary Jeremy Hunt said that the "tragic" incident showed that the health service should "never take safety for granted".
Speaking at the NHS Confederation annual conference in Liverpool, Mr Hunt said: "The truth is that we have come a long way (in patient safety) but we have a lot further to go.
"Today's story about the tragic blood poisoning of 18 children shows we can never take safety for granted. It also shows the importance of prompt and early identification of problems."
The manufacturer of the fluid has said that the suspected contamination has been traced to a "sourced" single raw material ingredient.
But ITH Pharma would not be drawn on details of the material it received from one of its suppliers.
Speaking outside the company's north west London base, ITH Pharma managing director Karen Hamling said: "From investigations carried out so far, it would appear the potential contamination is linked to a single sourced raw material ingredient.
"As a mother, as a pharmacist , as someone who has worked for 30 years in healthcare, inside and outside the NHS, I am deeply saddened that one baby has died and others have fallen ill from septicaemia."
The incident is being investigated by officials from the Medicines and Healthcare products Regulatory Agency (MHRA). The authority is also promoting the ITH Pharma 's product recall on certain batches of the fluid.
Ms Hamling added: "We have instituted a recall of the limited number of batches which could potentially have been affected and all stock has been removed from circulation. Given this action, there is no reason for patients, their families or healthcare professionals to be concerned."
She said the MHRA was on site "all day yesterday" and no restriction has been placed on their manufacturing licence.
One of the latest cases was confirmed at Peterborough City Hospital in Cambridgeshire and two probable cases have been identified at Southend University Hospital and Basildon University Hospital, both in Essex, PHE said.
Four cases of septicaemia, which occurred after the youngsters were infected with the bacterium known as bacillus cereus, were identified at the Chelsea and Westminster Hospital and one at the Whittington Hospital, both in London; three at the Royal Sussex County Hospital in Brighton; two at Addenbrooke's Hospital in Cambridge; and two at Luton and Dunstable University Hospital in Bedfordshire.
Another three cases were confirmed in babies being treated at St Thomas' Hospital in London, which is where the baby died on Sunday after being infected by the suspected contaminated intravenous fluid.
It is understood that all of the other cases are responding to treatment with antibiotics.
Earlier it emerged that the suspected contaminated batch of a food supplement was sent to 22 hospitals, according to an alert on the MHRA's website.
It was sent to 10 hospitals across London: Chelsea and Westminster, Hillingdon, the Harley Street Clinic, Homerton, King's College, the Portland, Royal Brompton, St Thomas', St George's and the Whittington.
The suspected contaminated batch was sent to four hospitals in East Anglia: Addenbrooke's Hospital in Cambridge, Basildon Hospital, Peterborough Hospital and Southend Hospital.
Two affected hospitals in the West Midlands are Birmingham Heartlands and Russell's Hall Hospital in Dudley.
The suspect batch was also sent to Lister Hospital in Stevenage, and Luton and Dunstable Hospital. Royal Sussex County Hospital and St Peter's in Chertsey were also sent the potentially faulty batch.
It was also sent to Stoke Mandeville Hospital in Buckinghamshire and Alder Hey Children's Hospital in Liverpool.
PHE's Dr Deborah Turbitt told Good Morning Britain that the organisation is "fairly confident" that the problem has been contained, saying: "We think we know all of the babies who have been affected at the moment.
"It is just possible that one or two babies have been infected and have been treated who have not been notified to us. We are confident that we know where this product has gone and all of the hospitals have been notified.
"We are fairly confident in talking with the company that manufactured the product that there was a single incident that happened on a single day to a limited number of products that went out to babies."
Bacillus cereus is a bacterium found in dust, soil and vegetation. It produces very hardy spores which in the right conditions can grow and create a toxin which causes illness, and it is likely to be on most surfaces.
An ITH Pharma spokesman said: "We are still using the same ingredient in new batches, from the same supplier."
He added they are not still using the same batch of products that is believed to be affected while the batches are created to order, for a single day's use and have a seven-day shelf life.
The potentially-contaminated batch was created on May 27 and any stock in hospitals should have been given to patients within seven days or discarded.
ITH Pharma is inspected every three to five years. It passed its last inspection which took place in April 2012, the spokesman said.
They also claimed that its workers undergo a "rigorous and continuous" training programme, recognised by the MHRA, which is led by an in-house team.
The spokesman said: "Employees are trained for up to six months on-the-job before they are able to handle products unsupervised. They have to complete and pass a test to our satisfaction in order to pass the training course. They are then tested every week throughout their employment with the company.
"Workers in the Clean Rooms are not allowed to wear make-up or jewellery."