The charity told Westminster planners the gardens were “a highly sensitive location” and that the structure “would have significant harmful impacts and fundamentally change the historic character and associated vistas in and out of the park”.
“The structure will dominate the park and eclipse the existing listed memorials which are nationally important in their own right,” the letter said. One memorial there is of suffragette leader Emmeline Pankhurst.
The Imperial War Museum has backed plans to build the £50m memorial and learning centre on the land, which is government-owned.
The Royal Parks also said: “We would not wish to close such a large area, or the possibility of the entire park, to visitors for the three years of its construction.”
Its letter warns security measures would lead to queues and congestion.
The “sombre nature” of the memorial would change the nature of what is currently a relaxed park, it said.
A spokesperson for the Ministry of Housing, Communities and Local Government said the memorial would honour victims and survivors of the Holocaust and subsequent genocides and educate future generations on the importance of fighting prejudice and persecution.
“No location in Britain is more suitable for the memorial than Victoria Tower Gardens, alongside Parliament where decisions in the lead up to, during and in the aftermath of the Holocaust were made, and amidst prominent memorials commemorating the struggle against slavery, inequality and injustice,” he said.
“The proposals have been developed with great sensitivity to the existing context and character of the gardens.
“We will retain 93 per cent of the open public space, improve views over Parliament and the River Thames and provide a range of accessible seating and a new boardwalk along the embankment.”
The government says no trees will be harmed by the plans.