Lord Berkeley, the deputy chairman of an independent review into HS2 set up by Boris Johnson, said existing services between Birmingham and Manchester were adequate but that services between the northern cities were not as good.
Downing Street is understood to be considering delaying or scrapping the Birmingham to Manchester leg of the route after costs soared in recent years.
The Prime Minister is facing criticism from figures such as Andy Burnham, Labour Mayor of Greater Manchester, who has demanded he press ahead with the northern route.
Former chancellor George Osborne and ex-deputy prime minister Lord Heseltine also warned that axing the Manchester route would be a “gross act of vandalism” which would mean “abandoning” the North and Midlands.
Writing in The Times, they warned Mr Sunak: “Governments are remembered for what they build and create. Make this mistake and yours may only be known for what it cancelled and curtailed.”
But Lord Berkeley, deputy chairman of the Oakervee report into the future of HS2, told The Telegraph: “The Prime Minister needs to be bold and call a stop to the line north of Birmingham.
“He’s under pressure from Manchester to keep it but it’s the wrong pressure. The problem in Manchester is how do you get anywhere else?
“What is more important is connecting the network much better in the North, between Manchester and Leeds and between Manchester and Liverpool.
“If the line stops at Old Oak Common, then the money saved could be used to improve rail services in the North.
“You don’t need a new line between Birmingham and Manchester because there is already a decent service there. What you do need is new services around Manchester, Liverpool and Leeds.
“Politicians like to think that everyone thinks it’s important to get to London, but what they want is to get to school or work without having to use their car.
“Services from Birmingham to Nottingham are also terrible – money could be spent to improve those if HS2 is cut back.”
Grant Shapps, the former transport secretary who is now Defence Secretary, said it would be “crazy” not to review plans for HS2 as costs have soared.
He said the Ukraine war and a spike in inflation mean any government would need to make “serious decisions” on affordability.
He told the BBC’s Sunday with Laura Kuenssberg programme: “We do have to respond to the budgets.
“We’ve not only been hit by the coronavirus, but the war in Ukraine ... any responsible government has to ask whether that sequencing still stacks up for what the country requires.”
“Every government has to… say ‘is this affordable under current circumstances?’”
Defence Secretary Grant Shapps says the government would be “crazy” if it didn’t reconsider the costs of HS2 after the Covid pandemic and the war in Ukrainehttps://t.co/Y0UVZDpMXq #BBCLauraK pic.twitter.com/L4auvqWYQY
— BBC Politics (@BBCPolitics) September 24, 2023
He added that when previous commitments had been made “no one knew we’d be in a war in Europe right now with all of the consequences, all of the costs, and all of the inflation”.
“And any government that doesn’t go back and then look at it is crazy,” he said.