Footage on social media shows demonstrators tearing the figure of Edward Colston from its plinth during protests in the city centre. In a later video, protesters are seen dumping it into the Avon at Bristol Harbour.
The demonstrations came in response to the death of George Floyd in Minneapolis on 25 May.
Home secretary Priti Patel said the toppling of the statue was “utterly disgraceful”, ”completely unacceptable” and “sheer vandalism”.
Avon and Somerset Police have launched an investigation to identify those involved in the incident.
“The vast majority of those who came to voice their concerns about racial inequality and injustice did so peacefully and respectfully,” said Superintendent Andy Bennett.
“However, there was a small group of people who clearly committed an act of criminal damage in pulling down a statue near Bristol Harbourside.
“An investigation will be carried out to identify those involved and we’re already collating footage of the incident.”
The controversial bronze memorial, made by sculpture John Cassidy, had stood in the centre of Bristol since 1895.
More than 10,000 people have signed a petition calling on Bristol City Council to have it removed.
Before it was pulled down, protestor John McAllister, 71, removed the black bin bags hiding the statues.
Mr McAllister said: ”It says ‘erected by the citizens of Bristol, as a memorial to one of the most virtuous and wise sons of this city’.”
“The man was a slave trader. He was generous to Bristol but it was off the back of slavery and it’s absolutely despicable. It’s an insult to the people of Bristol,” he added.
Colston worked for the Royal African Company and later served as the Tory MP for Bristol.
The slave trader is still a large presence in the city of his birth, where many schools, buildings and charities are named after him.
Labour MPs tweeted in solidarity with the removal of the statue, with Clive Lewis writing: ”If statues of confederates who fought a war for slavery & white supremacy should come down then why not this one? Someone responsible for immeasurable blood & suffering. We’ll never solve structural racism till we get to grips with our history in all its complexity.”
Dawn Butler, MP for Brent Central, used a clenched fist emoji and said “this caused me to exhale”.
On Sunday, thousands of people joined anti-racism rallies in Bristol and other cities across the UK. One protest against police brutality took place outside the US embassy in London.
Additional reporting from PA