The TV presenter stepped down from the ITV daytime after more than two decades following reports of a fall out with former best friend and co-host Holly Willoughby.
He then quit the channel completely after admitting to having an “unwise, but not illegal” affair with a younger male colleague.
Prior to this, actress Margolyes, 82, had become a semi-regular feature on show, with Schofield and Willoughby often describing her as one of their favourite guests.
The Harry Potter star expressed her sadness at the situation as she discussed her enjoyment of appearing on daytime TV – where she says “egos are less in evidence” – in her new memoir Oh Miriam: Stories From An Extraordinary Life.
She writes that she found Willoughby and Schofield to be “warm, welcoming and ready to join in the frolic”, adding: “I feel sad I shan’t have the chance to make Phil laugh again; he was a darling then and to me he always will be.”
Meanwhile, her most recent appearance on This Morning threw the studio into absolute chaos when she swore live on air twice, forcing Willoughby and co-host Josie Gibson to apologise.
She also didn’t hold back when talking about her sex life and detail getting her vagina steamed during a recent trip abroad.