LSE ‘verging on irresponsible’ for claiming low emission zones improve school grades

London School of Economics report challenged
London School of Economics report challenged - MARK PHILLIPS/ALAMY

The London School of Economics (LSE) was “verging on irresponsible” to suggest that the capital’s low emission zone (LEZ) had boosted students’ exam results, academics have warned.

Last week, the LSE published a working paper that said researchers had found a “meaningful improvement” in key stage 2 (KS2) test results for 10 and 11 year-olds in Greater London, compared with pupils in 20 other cities in England.

The team concluded that the upswing was caused by falling emission levels, brought about by the LEZ that charges polluting vehicles.

Researchers from the Grantham Institute at the LSE said the effect was “similar to reducing class size by 10 students or paying teachers large financial bonuses, and almost equivalent to significantly increasing average teacher quality”.

However the findings have been criticised by some academics who said there was no evidence of a causal link and that other factors could be responsible for the disparities between pupils in different parts of the country.

Prof Kevin McConway, the emeritus professor of applied statistics at the Open University, said: “What I do think is verging on the irresponsible is press releasing it, with a release that uses language of cause and effect without expressing any doubt that the change in KS2 test scores was caused by the LEZ.

“The article itself is also strongly causal in its wording, and the authors claim that their methods establish that the association is indeed one of cause and effect. Obviously it’s possible that it is cause and effect.

“But I don’t believe that the analysis in the paper establishes this anywhere near firmly enough to make the claims that are made in the paper and the press release.”

The Grantham Institute paper was peer reviewed internally and has been submitted for publication.

Dr Sefi Roth, associate professor of Environmental Economics
Dr Sefi Roth, associate professor of Environmental Economics

In a commentary, Sefi Roth, associate professor of Environmental Economics and the study’s author, said the team was able to “go further than identifying a simple correlation to describe compelling evidence for a causal effect”.

He said: “We did this by using state of the art econometrics techniques to deal with possible confounding factors, such as the fact that London is inherently different to other cities.”

However, experts said they were unconvinced the data was strong enough to come to such a definitive conclusion.

Dr Gavin Stewart, a reader in interdisciplinary evidence at Newcastle University, said: “They’re making very bold claims off limited data.

“I’m not as convinced by the analysis as the authors are. It seems unlikely that the effect magnitude associated with air pollution would be larger than other drivers, notably socio-economics and educational funding.

“To believe these results I’d want a whole set of school level covariates about staff student ratios, funding per pupil etc and a whole set around pupil socio-economics.

“The conclusion should be much more circumspect without robust evidence that the primary drivers of educational attainment aren’t operating. The result could easily be spurious association or driven by regression to the mean.”

He added: “Irrespective of whether it has been through peer review this work does not present sufficient evidence on uncertainty to be reliable in terms of strength of evidence in my judgment.”

A number of academic papers about the capital’s controversial road policies have been criticised amid claims they have been subject to political interference, and used to push unpopular agendas.

Sadiq Khan, the Mayor of London
Sadiq Khan, the Mayor of London - MARK KERRISON/ALAMY

Last year, The Telegraph revealed how the office of Sadiq Khan, the London Mayor, tried to discredit and “silence” scientists who found that his ultra-low emissions zone (Ulez) policy had little impact on pollution.

Some experts said the link between educational achievement and pollution was plausible, but warned that other factors could also be causing the effect.

Commenting on the LSE paper, Dr Stephen Burgess, the group leader at the MRC Biostatistics Unit, at the University of Cambridge, said: “Is this plausible? Yes. But there could be other explanations as to why the changes would be sharper in London than in other cities.

“If the difference were sharply focused on a narrow time window, then it can be hard to find an alternative explanation. But here, the excess difference could occur for a number of reasons.

“A plausible example is immigration. If patterns of immigration were different in London from non-London cities across the time period, this could account for the difference in differences.

“I don’t want to nail a mast to immigration here – there are surely a dozen plausible factors you could think of by which London differs from the other cities. The authors have accounted for some of these in their analysis. But it’s tough to account for every possible alternative explanation.”

The LSE paper notes that the Greater London LEZ was introduced in 2008 and has “evolved into one of the largest and most rigorous traffic regulation policies”.

A Grantham Institute spokesman said: “The paper was reviewed by two senior researchers at LSE who were not involved in the work and we are in the process of submitting it for publication in a leading economics journal.”

The new research was welcomed by the London Mayor’s office.

A spokesman for the Mayor of London, said: “We know that air pollution is extremely dangerous for children due to the long-lasting impact it can have on their health and life chances.

“That’s why the Mayor has taken bold action to clean up the air across London. Everyone has a right to breathe clean air and this new academic report provides another clear example of the positive impact low emission zones can have.”