May’s time is up. She must make way for a caretaker prime minister

There is an unforgettable moment in Richard Pryor: Live on the Sunset Strip that comes unexpectedly to mind in these strange and desperate times for our country. In it, the comedian recalls an encounter with his close friend, the legendary NFL running back Jim Brown.

Pryor is in a wretched mess, confined to his room, freebasing cocaine with a pipe that he imagines is whispering to him. Brown is unimpressed. He simply asks: “What you gonna do?” Again. And again. And again. In March 2019, the UK is Pryor – strung out, in this case, on Brexit – and the rest of the world is Jim Brown. Everyone can see that this is a country on the edge of nervous collapse.

Theresa May’s deal with the EU has been twice rejected, heavily, by the Commons. Her cabinet is a spider’s web of plots and schemes. A million marchers took to the streets in London on Saturday to demand a people’s vote. At the time of writing, 5 million had signed the digital petition to parliament to revoke article 50.

Meanwhile, Nigel Farage warns that, if the exit process is delayed for long, he will “tear [May’s] party limb from limb”. The very public trust that Brexit was supposedly intended to restore has been further disfigured by three years of failure. The Brexit pipe billows lethal smoke. What you gonna do?

Here’s what could be done. This is no time for wishful thinking – nor, however, is it a moment for self-indulgent fatalism either. At the risk of sounding portentous, this is an hour of civic obligation that should soar way above party politics.

First, May should announce her resignation now, immediately, without delay. Though I have been arguing that she should quit since the 2017 election, I was not persuaded that the present crisis would be alleviated by her departure – until, that is, last week.

By pitting parliament against the public in her horrendous speech on Wednesday, she disgraced the high office that she holds. Desperate and cornered, she rummaged in the populist’s toolkit and deployed the scuzzy rhetoric of the leader posturing as the friend of the people against the treacherous elites. This, remember, is the politician who once had the courage to tell the Tories that they were perceived as the “nasty party” and promised to address the “burning injustices” of our society. It has been a long fall.

As defects in a prime minister negotiating a complex international treaty, May’s inertia, lack of agility and fixation with Conservative party unity have always been serious problems. But to that is now added toxicity.

Resented by MPs, derided by the EU27, reduced to a Margaret Thatcher tribute act, she is now a clear and present threat to our chances of avoiding a no-deal disaster on 12 April. Every additional hour she remains in office chips away at the national interest.

Second: her replacement should be David Lidington, the Cabinet Office minister who is already her de facto deputy. Lidington – who has denied “any wish to take over” – should do so unchallenged, exactly as Michael Howard assumed the leadership of the Conservative party in 2003 after Iain Duncan Smith was sacked by his own MPs. But he must also do so on the explicit basis that he will himself announce his own resignation before, say, the Tory party conference opens in September. The Conservative tribe will indeed insist on a fullblown, no-holds-barred leadership battle some time soon.

It would be horrendously unpatriotic for the party to engage in that particular conflict at this particular moment. What the nation desperately needs now, especially in the next few weeks, is a caretaker prime minister to lower the temperature, create space for parliament to think and behave imaginatively, to encourage cross-party negotiations.

Neither Michael Gove nor Jeremy Hunt, the other two names mentioned by Tory MPs as potential short-term successors to May, is interested in being a caretaker. Both men want the job for real, and have already started to take soundings among their colleagues. With all due respect, Lidington is not a serious candidate for long-term tenancy of No 10: which is precisely why he is the ideal candidate for this unique context. Yes, he is a remainer. But he is also a decent man who would listen to everyone, and try to get us all over the line without catastrophe. That’s no small consideration.

Third, as parliament moves centre-stage – which it will this week – the main parties must unambiguously offer MPs the right to vote freely on the different options with which they will be presented. Last week, Kwasi Kwarteng, the Brexit minister, conceded in the Commons that this was indeed the logic of the case.

But chancellor Philip Hammond was much more hesitant on this particular matter in his interview with Sophy Ridge on Sky News on Sunday. Meanwhile, shadow Brexit secretary Keir Starmer told the BBC’s Andrew Marr that Labour MPs should expect to be whipped when an opposition “policy position” was at stake – which covers just about everything.

As hard as it will be for the party managers to let go, they must do so if this exploratory exercise is to mean anything at all. It has become a cliche that parliament knows what it does not want – May’s deal – but is nowhere near a consensus on what it would prefer instead. To stand even a chance of reaching such a consensus, all 650 MPs must be free to speak their mind, and to think as public servants rather than lobby fodder. We have seen where the flailing efforts at command and control have taken us. It’s time to give creativity a chance.

Related: Brexit petition to revoke article 50 hits 5m signatures

Fourth: hardest of all, parliament must come up with a workable alternative to May’s deal. There are endless assertions about the true state of the parliamentary arithmetic: usually, that the Commons, left to itself, is likely to back something like “Norway plus”, the arrangement whereby the UK would remain in both the customs union and the single market, perhaps with bespoke opt-outs. I am not yet persuaded by such claims. Why be a rule-taker when you cannot be a rule-maker? How many MPs would have the sense to back continued freedom of movement? Still: I would definitely like to learn the truth of the matter.

I would like to know where the critical mass of unconstrained parliamentary opinion really lies, and then see the government given binding instructions to return to Brussels with a mature plan: a new blueprint, to be implemented subject to the EU’s approval and a public vote in this country. The odds, I readily accept, are still against the smooth enactment of this four-phase strategy. Demons have indeed been unleashed by Brexit to wage war with reason and decency.

So all one can say with confidence is that this truly is one of those moments that occur quite rarely in the life of a nation; when all MPs – all of us, in fact – must summon the courage to look in the mirror and ask: what you gonna do?

• Matthew d’Ancona is a Guardian columnist