The song contest is being held in the UK in 2023 and there has been much speculation about who might host the event.
But Norton - who provides the Eurovision commentary for viewers - has suggested it won't be him.
"Well, that's your job surely?" replied Willoughby.
But Norton replied: "I don’t want to give up the commentating.
"If I stop the commentating someone else will do it and they might be better than me and then I’ll lose that job."
Schofield asked whether the "conversations had been ongoing".
"Well, I might wander on and wave, and then run back to my rabbit hutch and put my ear things on," Norton joked.
The UK has been given the opportunity to host Eurovision in May 2023 after finishing second in the competition this year, with Sam Ryder as the entry.
Ukraine actually won, so traditionally the contest would go there the following year.
However, due to the ongoing situation with Russia, the UK will host.
Several cities were in the running to host but it has now been whittled down to Liverpool or Glasgow.
Glasgow is a strong contender, with its 14,500 seat OVO Hydro arena. And if Liverpool were picked, Eurovision would take place at the M&S Bank Arena, which holds 11,000 people.
The BBC has said that its final decision on where it will take place will be made within weeks.
Watch: Graham Norton celebrates wedding with lavish Irish party