Why are we in thrall to 'superforecasters' like Professor Neil Ferguson?

"And that," said Prof Neil Ferguson, epidemiologist extraordinaire, oracle of Imperial College London, lockdown luminary and our pandemic prognosticator-in-chief, “is where the crystal ball starts to fail.”

He was talking a mere 10 days ago, from the pulpit of that Delphi de nos jours, national television. It was “almost inevitable”, he said, that the country would soon reach 100,000 daily Covid cases. Only when it came to speculation that infections could double that, at 200,000 a day, did cracks emerge in the crystal ball. “It’s much less certain,” he said. Still, 100,000 was basically locked in.

Not today. Today it’s clear that almost from the very moment that Ferguson told us to batten down the hatches and prepare for a storm of cases, blue skies broke out and case numbers began to plummet, now standing at about half what they were when he looked into his crystal ball with such confidence.

Of course, it’s possible that July 19’s unlocking has not yet had time to feed though into transmission, sickness and, potentially, deaths. Ferguson may yet be right.

But there are experts who take a hard line with his mid-July estimates. One of them, it’s hard not to notice, is Ferguson himself. Only on Tuesday the good prof took to the radio to say that “infections in the community are plateauing. I’m positive that by late September-October time we will be looking back at most of the pandemic”.

Who is right? Mid-July Ferguson or late-July Ferguson? It’s hard to be sure. Making such predictions is clearly astonishingly difficult. Ferguson himself acknowledges as much: “We have to monitor quite a number of different indicators to see what’s going on,” he says. But then why use words like “inevitable”?

Even other forecasters, like the American Nate Silver (who had his own fingers badly burned by declaring Hillary Clinton a shoo-in over Donald Trump), have had enough. “I don’t care that the prediction is wrong, I’m sure this stuff is hard to predict,” he wrote of Ferguson’s latest flip-flop. “It’s that he’s consistently so overconfident.”

In most walks of life, being made to eat your words so publicly and so swiftly might carry a cost. But Ferguson doesn’t see a problem. “I’m quite happy to be wrong if it’s wrong in the right direction,” he said. By which he meant overestimating by far how bad things would get. Which sounds, frankly, just a little smug.

It’s all terribly confusing, this world in which experts are hauled onto the news to tell us what is going to happen only to contradict themselves a few days later. In this way Ferguson has become the contemporary equivalent of the economist John Maynard Keynes (who infamously predicted we’d be enjoying a 15-hour working week by now). In 1961, asked by US Congress to report on the “Impact of Automation on Employment”, Nobel-winning economist Paul Samuelson noted the difficulty of such predictions. “If Parliament asked six economists for an opinion on any subject they always got seven answers,” he noted. “Two from John Maynard Keynes.”

Holding multiple, apparently contradictory views, is no bar to greatness, such stories tell us. Which means that neither is being wrong. And Ferguson is no stranger to that, either.

If, as he latterly suggested, we are now in Covid’s death throes, then his quick about-turn on the matter will become a fitting bookend to his pandemic prognostication.

Its pair was that famous infection model of March 16 2020, blandly titled “Report 9”, which appalled politicians, shocked the country into lockdown, yet turned out to be based on computer code that software engineers described as “horrible” and “ a buggy mess”. Though it was later agreed that the code was just about functional, its assumptions – that if nothing was done 81 per cent of the population would be infected and 0.9 per cent would die (more than half a million people in Britain) was never the subject of consensus. Far from it. Its key assumptions were hotly debated. Other academics suggested they might be “substantially inflated”.

Imperial’s predictions came under global scrutiny, too. One study, which later compared eight worldwide prediction models noted that Imperial’s contained “larger errors... largely driven by the tendency to overestimate mortality”.

Not that Ferguson was always too gloomy. In March he suggested he was “80 per cent sure” that we would all be enjoying our holidays right now, as case numbers would be “very low” by summer. Typically, on the one occasion he was upbeat, his worst fears were realised with the emergence of the delta variant. Usually though, he has, shall we say, erred on the side of the Grim Reaper, as regards bird flu, swine flu and foot and mouth.

So why, after this record of debate, dispute, contradiction and correction, are we so in thrall to the modellers?

It can feel like this search for “truth” – and an infatuation with those who claim to deliver it – is an epidemic of its own. Sages of today’s business world are agog with forecasting. “In virtually every decision they make,” says the Harvard Business Review, “executives today consider some kind of forecast. Sound predictions of demands and trends are no longer luxury items, but a necessity.”

Inevitably, then, a market in forecasting has developed, because not all forecasting is the same. A plethora of books now show us that the very fact of being human, with all our instincts and biases, is inimical to “good” forecasting. So-called “superforecasters”, who manage to look beyond all that, are the exception, though their skills can be learnt, notes the US academic Philip Tetlock (who knows more about them than anyone).

Much of the world – from financial markets to intelligence agencies – is in the business of forecasting. Dominic Cummings devoted himself to the better deployment of data in government forecasting. With good reason: Tetlock has written a book about superforecasting, and reckons the average forecaster is “roughly as accurate as a dart-throwing chimpanzee”.

Yet that does not – has not ever – put us off. Whether you take your predictions in the form of tea leaves or chicken entrails, the I Ching or the Les Propheties de Nostradamus, CIA analysis or stock tips, mankind has had a soft spot for seers since the dawn of time.

Today, you might think, there should be no excuse for getting predictions wrong – given the deluge of both numbers and computers to crunch them. But it turns out that algorithms can be biased too, when they are fed fallible data by fallible humans.

Today we listen to forecasts of disaster as never before, the worse the better, moderation be damned, and drink down every word. Who could have predicted that?