Denise Fergus relived the day she lost the toddler in a new documentary with Sir Trevor McDonald aired on ITV on Thursday at 9pm.
She recalled: “He was out of his buggy – James was never out of his buggy so he felt a bit of freedom.
“I let go of his hand just for a moment to get my purse out of my bag.”
The interview marks 25 years since two-year-old James was abducted and killed by 10-year-olds Jon Venables and Robert Thompson.
After James went missing, Ms Fergus said she ran from shop to shop asking people if they had seen her young boy.
She said: "I was that distraught that one of the staff said to me calm down, come and have a cup of coffee and I said to him a cup of coffee isn’t going to help me get my son back."
Ms Fergus invited Sir Trevor to her home for the frank interview where she discussed the case, including the day she buried her son.
She said: "That was one of the hardest days of my life. Burying a child, no parent should have to do that.
"You just think to yourself, it should be me in that casket not you."
She shared memories, family photos and videos, and told how she still sets a place at the table for James, who she remembered as "so loving".
"He was always laughing. He love listening to music. He was just a lovely little boy," she said.
Ms Fergus also opened up over her anger at the sentence the two boys received for killing James.
The pair were only jailed for eight years and were granted new identities and lifelong anonymity when they were freed in 2001.
Ms Fergus said: "I thought eight years for taking a child, for murdering a child - is that it?"
She added: "All the fighting that I've done all these years, I've never ever had justice for James, ever."
The programme featured testimonies from other important figures in the case, who were interviewed in James’s home town of Liverpool.
Venables was returned to prison in November, having been charged over indecent images of children for a second time.