The southwest Pacific archipelago voted on whether it wanted independence from France following a three-decade decolonisation effort.
The "no" camp won 53.26% of the vote, after provisional results earlier showed it was edging ahead.
It means New Caledonia will now remain a French territory.
More than 180,000 voters were asked: "Do you want New Caledonia to gain full sovereignty and become independent?"
Earlier on Sunday morning, partial results released by France's ministry for overseas territories showed the "no" vote had 52% support with 226 out of 304 polling stations counted.
The independence referendum was part of a process aimed at settling tensions between native Kanaks who want independence and those who want to keep ties with France.
The vote was the final step of a process that started 30 years ago following years of violence between the two groups.
The Kanaks had once suffered from strict segregation policies and the vote comes at a time when the legacy of colonialism is under scrutiny across the world.
A similar referendum took place two years ago, when 56.4% of voters chose to keep the status quo.
Independence supporters wanted all sovereign powers - including justice, police, military, currency and foreign relations - to be transferred from France to New Caledonia.
New Caledonia became French in 1853 under Emperor Napoleon III and for decades was used as a prison colony.
After the Second World War, it became an overseas territory and all Kanaks were granted French citizenship in 1957.