The decline in train performance has increased in recent months, according to new figures.
Only 70.2% of trains in Britain arrived on time in September, compared with 72.6% in April, reported the Office of Rail and Road (ORR).
Punctuality has declined since the highs recorded during coronavirus lockdowns, when passenger numbers plummeted.
The performance of freight trains is also at its lowest in the past five years, the ORR said in a letter to infrastructure management company Network Rail.
The ORR highlighted key areas of improvement including better track reliability in the North West and Central region, greater resilience of overhead lines in the Eastern region, and the renewal of track on the Thameslink route that connects north and south London.
John Larkinson, chief executive of the ORR said: "Passengers and freight are suffering from poor train performance, with issues extending across all of Network Rail's regions.
"There are of course factors beyond Network Rail's control to delivering good train performance, including further industrial action and potential extreme weather events.
"And there are areas such as trespass and theft where Network Rail has worked hard to reduce delays. But it can nevertheless do more.
"It is essential that the company now delivers on the specific interventions we have set out today.
"We will continue to scrutinise delivery and will take further action if there is insufficient progress."
A spokesman for Network Rail said: "While train performance remains well ahead of pre-COVID levels, there is no doubt that a long, hot, dry summer, accompanied by strikes, industrial unrest and a fall-off in infrastructure reliability is taking its toll.
"We recognise the issues and problems at the root of this and are working hard to make improvements and provide both our passengers and freight users a better service they can rely on."