The Olympic swimmer and his professional dance partner Katya Jones found themselves in the bottom two in Halloween Week alongside Loose Women star Judi Love and Graziano Di Prima, who were eventually sent home.
Peaty, 26, told Strictly: It Takes Two: "It's unfortunate that we ended up in the dance off. I mean, I'm still carrying a bit of baggage from the weekend because it was tough to go to the dance off.
Read more: Motsi Mabuse dismisses Strictly fix claims
"And when the red light comes on, you're like, 'Oh no I've gotta go straight over to Tess and talk about it'."
Watch: Who is Strictly's Adam Peaty?
Referring to his score he went on: "Because it was so unexpected because I thought, 'You know what? 29 solid.'
"But it just shows you no one is safe in this show, and I've got to find my confidence again now in this Jive and go, 'You know what, I do deserve to be here... I think'."
Peaty and Jones were awarded a score of 29/40 for their Viennese Waltz by the judges — placing them in the middle of the leader board.
Voting figures leaked to the Mail On Sunday showed that the previous week the Olympian and his partner had been the third most popular couple, winning a 16.4 per cent share of the vote. That was for raunchy Samba to George Michael's Faith.
Their ghostly Viennese Waltz to Arianna Grande's Dangerous Woman, was criticised by head judge Shirley Ballas because Peaty's bottom was "pushing out slightly".
Peaty laughed to It Takes Two host Rylan: "I struggle to tuck it I'm gonna shake it as much as I can!"
Jones added: "It's his biggest asset in Latin, and it gets in the way in ballroom!
"But we have been working on it, of course. We take every judge's comment on the chin."
After being saved by the judges in the dance-off on Sunday, Peaty admitted he was feeling "humbled".
He wrote on Instagram: "Truly humbled to be able to continue on this show and I will come back stronger.
"Unfortunately this week it meant I was going against @1judilove and @grazianodiprima who are incredible competitors and even more incredible people.
"Thankyou to everyone who has supported me so far, thankyou to @katyajones for being an amazing choreographer and person and pushing me bound my boundaries! I’m so grateful."
This weekend the pair will be dancing a "fun" Jive to Little Bitty Pretty One by Frankie Lymon & The Teenagers.
Watch: Who's judging this year's Strictly?