The 78-year-old choreographer holds the record as the oldest ever contestant to take part in the ITV reality show - and confessed it was the first time she had ever been camping in her life.
Dame Arlene told hosts Ant McPartlin and Declan Donnelly she felt relieved to be voted out of the castle, as she wants to see her family again.
But admitted that being first to leave was "Emotional because of everything this has been. It's been something in my life I never imagined doing and its been extraordinary."
The theatre director admitted her experience on the show had been "Much harder" than she expected.
Talking about why she had signed up to take part she confessed: "I never camp - my family all camp, I've never camped. I've done it now."
Dame Arlene said her inspiration to sign up for the challenge - which saw her take part in a trial in which she was locked in a coffin filled with snakes - had been 13-year-old Lily Douglas. The teenage dancer has been battling Ewing’s Sarcoma cancer for the last four years and recently took part in a charity walk for the NHS, winning her a Pride Of Scotland Award.
Dame Arlene described her as, "One of the bravest and fearless girls I know."
She went on: "I thought here am I afraid of so many things, what better way to push myself?"
The former dancer admitted she had bonded with music producer Naughty Boy during her time in the castle.
She said: "I did look after him, we have a bond, music and dance. He's very special, a very special human being."
And Dame Arlene said she hopes Naughty Boy will go on to win the show.
TV presented Richard Madeley became the first celebrity to leave the camp at Gwrych Castle in north Wales after just five episodes. He was forced to quit the reality show after he fell ill and was sent to hospital for a health check - meaning he had broken the camp's COVID bubble and was not allowed back in.
Last weekend the ITV reality show was cancelled and the celebrity contestants - who include footballer David Ginola, Loose Women presenter and former Saturdays singer Frankie Bridge and Emmerdale star Danny Miller - were removed from the castle for their own safety following Storm Arwen.
The contestants were quarantined in hotels individually, in line with coronavirus restrictions, and with no access to internet, TV or phones, while the damage to the production site was repaired.
They returned to the castle on Tuesday 30 November after three days away.
Watch: Who is Dame Arlene Phillips?
The show's final is scheduled for Sunday, 12 December from 9pm to 10.40pm, when we'll see the latest Queen or King of the Castle crowned.
I'm A Celebrity returns at 9pm on ITV on Monday, 6 December.