The 40-year-old former British soldier and father-of-five, who served four months in prison in 2013, said that when he was released he was so impoverished he could not afford milk for his family.
Middleton told Eddie Hearn’s No Passion, No Point podcast: "I remember when I got out of prison, I remember opening the fridge and literally having no milk for the kids and I had to get my daughter's penny jar and smash it open and get all the 20 pences and all the 50 pences.
"I had to literally accumulate all those to get two pints of milk until I waited for my next pay cheque and I remember thinking, 'We can't live like this, I've got so much to give.'
"I was a special forces operator, I know I've got a resilient and robust mindset and unique way of thinking so this can never happen again.
"And then I went from that to writing my first book, more TV shows, more books, doing UK tours.”
He added: "I've had no money and now I have money and I know which side I'd rather be."
Middleton served four years of a 14-month sentence for unlawful wounding and common assault on two police officers outside a nightclub in Essex.
He said being convicted after the altercation was a relief.
Middleton said: "I can just remember as soon as that hammer went down, I remember this weight being lifted off my shoulders because I knew what I needed to do – I've got to do 14 months.
"The worst moment is not being able to own a situation, I didn't know if I was going to get one year, two years, five years. When you are in control you can take ownership."
The TV star met wife Emilie in 2004 and they married in 2006 when he was still serving in the British army.
They have four children together: daughters Shyla, 13, and Priseis, 5, and sons Gabriel, 11, and Bligh, 4.
Middleton also has a son from a previous relationship.
He joined the army at the age of 17 in 1998 and served for 14 years. He is now known for interrogating celebrities in Channel 4 reality show Celebrity SAS: Who Dares Wins.
Watch: Ant Middleton interrogates Katie Price