The Infrastructure Projects Authority (IPA) gave Phase 2b of the route – between Crewe and Manchester and the West Midlands and Leeds – a 'red' classification, meaning it was deemed unachievable in terms of time and budget.
It comes after West Midlands Mayor Andy Street said it was a "huge missed opportunity" that HS2 trains heading north would not pass through Birmingham's central Curzon Street station until Phase 2b was complete.
The IPA considered HS2 in its latest major projects report, which rated the viability of dozens of national infrastructure schemes.
It said the red rating meant that "successful delivery of the project is unachievable".
"There are major issues with project definition, schedule, budget, quality and/or benefits delivery, which at this stage do not appear to be manageable or resolvable.
"The project may need re-scoping and/or its overall viability reassessed."
Michael Fabricant, Conservative MP for Lichfield, said ministers should rethink HS2 on the back of the report.
He said: "I shall be asking for a statement from the Transport Secretary, Grant Shapps, regarding this assessment of the viability of the scheme and whether the Government now plans to abandon this leg of HS2.
"If that is the case, it weakens the whole case for the high speed line in Birmingham and damages still further its viability in the West Midlands."
Mr Fabricant has since tabled a written parliamentary question on the matter.
He has also called on Mr Street to accept that HS2 would "sideline" the West Midlands, with Birmingham being "merely a spur off the main north-south route".
A spokesperson for HS2 Ltd said the IPA's assessment was made last September ahead of the Oakervee Review of the project, and that HS2 Ltd's "capability" in delivering it had now been assured.
Mr Street has described HS2 as "vital" to the West Midlands economy and said it would create thousands of local jobs.