Pollsters got it wrong in 2015, so could Labour’s lead be overestimated?

There have been polls, so many polls it is almost impossible to keep up with them. But they are all showing extraordinary Labour leads, even two weeks into the campaign.

“If the polls are right” seems to be the soundtrack to the campaign. If the polls are right the Conservative party may end up facing its worst defeat since 1906. But could the polls be wrong?

No one really thinks so. Some think the Labour lead could be soft or unenthusiastic, that it may change during the campaign, but no one really doubts it is real.

Related: UK general election: the seats the Tories will lose if the polls are right

The polling industry has not changed how it collects data since 2019 – it was right then and it should be right now. But there is still that voice: “unless, unless …” In 1992 and 2015 the polls predicted a close contest, a possible hung parliament and yet the Conservatives won. Could the same things go wrong?

Polling companies want to get as close to the correct result as they can, so these previous misses led to reflection and changes in the industry. When a party is very unpopular people are less likely to tell pollsters they voted for it at the previous election or that they will vote for it at this one.

This so-called “shy Tory” effect, often blamed for the 1992 polling “miss”, has been tackled by weighting the samples to reflect past vote and likelihood to turn out – ensuring the sample used reflects the distribution of votes at the previous election and corrects for people who say they are unlikely to vote.

The British Polling Council found samples were to blame in 2015: the samples had too many Labour voters in them. This may sound obvious in a polling miss but there are ways for even a strong sample to go awry. The industry has worked hard to correct this, though there remain challenges among some groups – most notably younger men without a university degree.

It is unlikely these issues are in play for this election, not least because the polling industry is now so alert to them. So what could be different in 2024?

As the campaign begins, many people tell pollsters they are undecided. And at this election there is an unusual pattern to these responses. Almost one in five of those who voted Conservative in 2019 say they are undecided, while for those who voted Labour in 2019 it is less than one in 10.

Past voters are quite likely to vote again, so this poses a challenge for getting vote shares right. Broadly, there are two methods in use. The first is to exclude the undecided when calculating the final vote shares. This assumes undecided voters will eventually end up distributed across the parties in the same way as everyone else.

The second method uses other information about these voters to allocate them to a party, this could be how they voted at the previous election or other survey questions about the parties and leaders.

Which approach is best depends on the context – and often we cannot know this in advance. In 2017, reallocation worked well as undecided Labour voters went back to the party, but for this election Conservative voters are splitting in many directions.

When there are more previous Conservative voters that are undecided, the second method will tend to produce larger shares for the Conservatives as pollsters infer that they will vote Conservative again. Opinium’s polls for the Observer have used this method for some time and have consistently shown smaller Labour leads.

Over the past few days, YouGov – which had previously been one of the companies showing the largest Labour leads – updated its methodology for allocating “don’t knows” and estimating turnout, adopting the more complex approach. This change reduced the Labour lead (using the same sample) from 27 percentage points to 21 percentage points. This gives us a sense of the impact of these methodological choices but no reason to think the Labour lead is not real.

People have expected the polls to change as people tune in and undecided voters decide. But so far this has not been to the advantage of the Conservatives. The differences between polling companies have started to narrow, but on a larger rather than smaller Labour lead.

Despite the extraordinary Labour leads being reported, there is nothing to really cause us to think the polls could “miss” and yet I may still watch the exit poll on 4 July through my fingers just in case.

• Paula Surridge is deputy director at UK in a Changing Europe and a political sociology professor at the University of Bristol