Married judges Robbie Williams and Ayda Field announced earlier this month that they'd be quitting the ITV reality competition after only a single season because their busy schedules made returning "impossible".
One might think that two judges quitting would be a setback for Cowell's plans to revamp The X Factor with a celebrity format this year, but apparently not.
"It was no secret [what they wanted] was millions and millions of pounds," he explained. "I am going to be really honest with you – and this is no disrespect to Robbie or anyone else who has been on the panel – my focus this year is mainly about the people who are coming on the show to compete."
Cowell went on: “I’m probably talking myself out of a job here, but I think there has been so much emphasis on the judges. We’ve lost sight of the contestants.
"It got crazy, I never thought this would happen 20 years ago, when I started this show, that we would be talking more about the judges than the contestants, it’s complete nonsense.
"Somehow that’s the way it’s gone. So that’s why me and Rob were talking, and he could see where I was coming from. It’s about getting the right people to come on the show, that was it."
The X Factor boss also stressed that Robbie and Ayda's departure from the show hasn't impacted their personal friendship at all.
"We still went on holiday together recently, even though we knew about X Factor," Cowell said.
"I’m seeing him when I’m back in LA. We are doing coupled-up things together, we have become close friends. They were both such a joy to work with."
The X Factor is expected to return later this year on ITV.
('You Might Also Like',)