The comedian landed in hot water in 2018 when he was snapped locking lips with his pro dancer partner Katya Jones, when she was married and he was in a relationship.
Chatting to his campmate Babatunde Aleshe, the star admitted the controversy had an impact on his career.
He said: "Easily the most depressing month of my entire life, Edinburgh after Strictly.
"We get to Edinburgh, right, I think it’s a 200-seater, I’ve never not in Edinburgh sold out the first night.
“It doesn’t even occur to me that it’s not sold out. Just before I’m about to go on, my agent comes in and goes, ‘Just so you know, there’s like 30 people’."
The star, 36, went on: “To get told that when you’re just about to go on, horrific.
"It was at that point I was like, ‘Oh, oh’.”
“My career was dead, I was over," said the comedian.
"My agent said, ‘There’s nothing I can do. I’m sorry, I’ve tried. I’ve tried everything'."
Walsh said when that happened he realised he needed to start driving so that he didn't have to sit on public transport with everybody looking at him.
Walsh and Jones' kiss took place after an evening out when they were partners on the BBC ballroom contest.
The comedian was in a relationship with actor Rebecca Humphries at the time but they then split.
Jones broke up with her husband Neil Jones a few months later, but they said the incident was not the cause of their break up.
Walsh has said that he hopes to get a 'new start' by doing I'm A Celebrity.
As he entered the jungle he said some of the things he'd read about himself in the wake of the Strictly scandal had 'hurt'.
He said he'd like a "new start" and to "just maybe change opinions".
Watch: Matt Hancock says Liz Truss's political career is 'over'