The AFP news service estimated that Hollande had won by securing somewhere between52 and 53 per cent of the votes.
The country's 46 million voters voted at polling stations from 06:00 GMT on Sunday, after a bruising campaign that has seen both men label the other as a liar.
The first polls opened in overseas territories on Saturday and the last polls were to close in France at 18:00 GMT on Sunday.
Political speeches and new opinion polls have been banned since campaigning officially ended on Friday night, with the last polls published before the deadline showing Hollande having a slight lead over the president.
Our corresponded in Hollande's base in the town of Tulle said that there seemed to be a sense of certainty there that Hollande would win the election.
"They're pretty much - when you look around and talk to people - convinced that he's set to be the next president of France," said Andrew Simmons.
"There's that much optimism in the socialist camp right now. It's much more unified."
Sarkozy has spent the last week energetically campaigning, in an effort to woo over voters of the National Front's Marine le Pen , a far-right nationalist who was knocked out in the first round of voting but refused to endorse either candidate's campaign.
The AFP news agency reports that voter turnout is high, though not at record levels, at 71.96 per cent by 5 pm local time (1500 GMT).
Voter turnout in the first round was high, at around 80 per cent according to official figures, and both run-off candidates have called on citizens to make their votes count.
Al Jazeera's Jacky Rowland, reporting from Paris, said voter turnout there has been slightly lower than the national average of just under one-third, but that it was expected that more would show upt vote on such a decisive day.
With the gap between the candidates the narrowest since campaigning began, Sarkozy has vowed a surprise, while Hollande has cautioned against assuming he will be France's first Socialist president since Francois Mitterrand left office in 1995.
Hollande, who has billed himself as a consensus builder, voted in his provincial political heartland Tulle in central France, while Sarkozy was voted in Paris' 16th arrondissement.
"It's going to be a long day," Hollande told reporters gathered to watch him vote. "It's up to the French people to decide if it's going to be a good day."
Sarkozy declined to comment to assembled television crews, although our correspondent says the incumbent focused his energy on wooing supporters.
"Certainly in his last campaign rallies, President Sarkozy was as pumped as ever, really, giving every last drop of energy to those crowds of loyal supporters who turn out for those kind of events," said Rowland.
If he loses, Sarkozy will become the first French president since Valery Giscard d'Estaing in 1981 not to be re-elected for a second term.
France has a strict ban on publishing result estimates until all polls close, but foreign media websites are expected to publish predictions before then.
Those found to be breaking the reporting restrictions will face a fine of 75,000 euros ($98,153).
Hollande won the April 22 vote with 28.63 per cent of the votes to Sarkozy's 27.18 per cent, and both candidates have been fighting for the votes of those whose candidates failed to make the run-off.
Le Pen, who won almost 18 per cent in the first round, has said she will cast a blank ballot, and has called on her supporters to do the same.
Jean-Luc Melenchon , the Left Front's candidate, however, endorsed Hollande's candidacy and called on the 11 per cent of voters who backed him to vote to unseat Sarkozy. Francois Bayrou , the centrist candidate, has also backed Hollande.
Hollande will need a strong mandate if he is to implement his proposed programme to fight EU-driven austerity measures, while Sarkozy, who has been instrumental in formulating the EU's response to the debt crisis, has said that electing a Socialist would cause further market instability.