Hazard, who has scored 11 Premier League goals and created four others for the leaders this term, has been regularly linked with a move to Real, who are coached by the Belgian's idol Zinedine Zidane.
Standard Sport understands that Chelsea have no intention of selling their star man - who two years ago was named the PFA Player of the Year - with Hazard also believed to be fully committed to the Blues.
But despite this and the fact the 26-year-old is contracted to Chelsea until 2020, reports this week have claimed that Real Madrid will try to sign Hazard this summer.
And former Real president Calderon believes that it will be Hazard who will dictate whether he leaves Stamford Bridge for the Santiago Bernabeu - not Chelsea - and insists the Spanish giants have the financial might to be able to pay whatever they must to sign the winger.
"Chelsea aren't a selling club, so it wouldn't be easy, but as always it would depend on the wishes of the player," Calderon told talkSPORT.
"If he wants to leave, he will leave - that's always the point.
"It's all about the player. The player must say, 'I want to leave and go to Real Madrid', and that will be the moment they start negotiating.
"Chelsea will ask for a lot of money, I think that's what the situation will be at the end of the season.
"Chelsea know quite well how much has been paid in the market over the last few years, with (Cristiano) Ronaldo, (Gareth) Bale and (Paul) Pogba - £100million is now the bar.
"That bar has been set by those transfers, so I think that's how much a club like Chelsea would be asking for and the economy at Real Madrid is healthy - I don't think that wouldn't be a problem for them."