President Emmanuel Macron said the move was a "real revolution" as cucumbers, lemons and oranges are among 30 items that will no longer be wrapped in plastic.
Peppers, courgettes, aubergines and leeks are some of the vegetables covered by the new law, passed by French politicians last year and which comes into force from New Year's Day.
Larger packs of fruit and vegetables, weighing over 1.5kg, will be exempt from the ban as will chopped and processed fruit products.
Sellers of other varieties, such as cherry tomatoes or soft fruits including raspberries and blueberries, will be given longer to find alternative coverings but plastic packaging will be gradually phased out for all whole fruits and vegetables by 2026.
It is estimated that more than a third of fruit and vegetable products sold in France are wrapped in plastic and officials hope that the new restrictions could prevent a billion items of single-use plastics from being used every year.
Announcing the changes, France's environment ministry said there must be restrictions on the “outrageous amount of single-use plastic in our daily lives”.
The fruit and vegetable ban forms part of a wider effort by France to phase out single-use plastic products in many industries.
A ban on plastic straws, cups and cutlery, as well as polystyrene takeaway boxes had already been in place.
Later in 2022 public spaces will be forced to provide water fountains in a bid to drive down the use of disposable, single-use bottles of water.
Publications will have to be shipped without plastic wrapping, and fast-food restaurants will no longer be able to offer free plastic toys with meals.
WWF France said the move was a step in the "right direction" but called for further action to reduce plastic use across all areas of society.
Pierre Cannet, WWF France director of advocacy and campaigns, said: “We need to stay humble and vigilant by saying there is still a lot to do.
“We’re still very far from an economy without plastic, and from all the steps needed to eradicate plastics pollution.”