A Conservative peer has become the latest senior Tory to call on Nadhim Zahawi to consider his position.
Lord Hayward said the Tory Party chairman should think about "standing aside" while parliament's ethics watchdog investigates his tax affairs.
"We don't know what the timescales are for the inquiry, and I think that's key," he told Sky News.
"I think he should be considering whether he stands aside for the period of the inquiry."
It comes after former minister Caroline Nokes said there were "too many unanswered questions" over the tax row and said Mr Zahawi should "stand aside and let the investigation run its course".
The inquiry by new ethics adviser Sir Laurie Magnus is expected to focus on whether the cabinet minister breached the ministerial code with the estimated £4.8m HMRC settlement he made while he was chancellor, but it could extend to his prior tax arrangement and whether he lied to the media.
Rishi Sunak said there are "questions that need answering" but has come under criticism for resisting calls to sack him.
It is unclear what the prime minister knew about Mr Zahawi's dealings with the tax office when he appointed him to his cabinet in October.
Lord Hayward is also a polling expert, and advised previous Conservative governments heading into elections.
He said that the issues Mr Sunak is facing at the moment, including Mr Zahawi's tax affairs, are working in tandem with the ongoing strikes to "flatline" the Conservative Party's popularity.
Labour leader Sir Keir Starmer is likely to seize on the fallout from Mr Zahawi's multi-million pound settlement at Prime Minister's Questions on Wednesday.
During last week's PMQs, Mr Sunak said that Mr Zahawi had "already addressed this matter in full" as questions were raised about his tax affairs.
But this was before the cabinet minister released a statement admitting he paid what HMRC said "was due" after it "disagreed about the exact allocation" of shares in YouGov - the polling company he co-founded - an error he noted was "careless not deliberate".
Sky News understands that, as part of a settlement with HMRC, Mr Zahawi paid a penalty to the tax collector.
Sources close to Mr Zahawi have said he is "absolutely not standing down".
In a statement on Monday, he said: "I am confident I acted properly throughout and look forward to answering any and all specific questions in a formal setting to Sir Laurie."