Labour is calling for an investigation after Mr Zahawi admitted an error in his tax affairs that was settled after a multi-million pound pay-out to HMRC.
Mr McFadden, MP for Wolverhampton South East and Labour's shadow chief secretary to the Treasury, said: "It's difficult for Mr Zahawi because the problem with this is what we're being asked to believe.
"We're being asked to believe that he had this asset worth £27 million that he didn't really know was his, and therefore didn't really know tax was due on it.
"He was using pretty heavy legal letters to get people to keep quiet about this... threatening legal action, and then we're asked to believe that it's all just carelessness. That doesn't quite fit."
Mr McFadden also said the matter is a "test" of Mr Sunak's promise to turn a new leaf after Boris Johnson's scandal-plagued premiership, suggesting his pledge would be undermined "if this is just shrugged off".
Fellow Midlands MP Shabana Mahmood told the BBC's Politics Midlands show that she thinks there are questions to answer.
The Birmingham Ladywood MP said: "The story is not over. The scrutiny on Nadhim Zahawi's tax came when he became Chancellor of the Exchequer so I think it is in the public interest that we understand.
"I think there should be an inquiry and we do need to know whether Nadhim Zahawi broke the ministerial code for example, so I think there are questions still to answer."
However, Shropshire MP Mark Pritchard told the show that Mr Nadhim Zahawi had paid all his taxes and he didn't see any other perspective to the issue.
The Wrekin MP said: "I understand that a spokesman from his office said that all taxes that were due have been paid in full, paid in the UK, and really I don't see any other perspective that people can have.
"It couldn't be clearer - the taxes have been paid. Period."
Foreign Secretary James Cleverly insisted tax affairs were "private matters".
Mr Cleverly stressed that Mr Zahawi's error was "careless" and not deliberate, amid allegations he avoided tax and had to pay it back as part of a seven-figure sum to HMRC.
The embattled Tory chair released a statement on Saturday to "address some of the confusion about my finances", but it raised further questions, including whether Mr Zahawi negotiated the dispute while he was serving as Chancellor.
He did not disclose the size of the settlement - reported to be an estimated £4.8 million including a 30 per cent penalty - or confirm whether he paid a fine.
Mr Cleverly denied any knowledge of the details when pressed during media appearances on Sunday.
"I don't know more than is in his statement," he said.
Asked whether Mr Zahawi should reveal this information, the Foreign Secretary said: "People's taxes are private matters. I know that as politicians we, quite rightly, are expected to have a higher level of disclosure than perhaps other people might do.
"Nadhim has issued a statement where he has admitted that he made a careless error, that this is now resolved."
Mr Cleverly also evaded questions on what Rishi Sunak knew when he appointed Mr Zahawi as party chairman.
He suggested the Prime Minister does not discuss ministers' "external affairs" during appointments and that it is the Cabinet Office's role to conduct due diligence on his behalf.
Mr Cleverly said he was unable to answer questions on Mr Zahawi's tax affairs because he was carrying out his duties as Foreign Secretary during the week and then "having a bit of a rest and doing some shopping".
Asked whether Mr Zahawi will survive in his role until Prime Minister's Questions on Wednesday, Mr Cleverly said: "What else am I going to say other than yes, because he's a very, very effective minister."
Reports about Mr Zahawi's tax affairs began emerging when he was appointed Chancellor by Mr Johnson last summer.
He denied allegations that he avoided tax by using an offshore company registered in Gibraltar to hold shares in the YouGov polling company he co-founded.
Opposition parties have demanded an independent investigation as well as the publication of all of Mr Zahawi's correspondence with HMRC.