Mr Netanyahu failed to secure a majority with his natural religious and nationalist allies in elections on Tuesday, with the exit polls currently too close to call.
But addressing his supporters early on Wednesday, he refused to concede defeat and vowed to work to form a new government that excludes Arab parties.
The result has set the stage for a period of coalition negotiations that could threaten the Prime Minister's political future and even clear the way for him to be tried on corruption charges.
Initial partial results showed Mr Gantz's centrist Blue and White party tied with Mr Netanyahu's Likud.
While the results do not guarantee that Mr Gantz will be the next prime minister, they signalled that Mr Netanyahu, who has led the country for over 10 years, could have trouble holding on to the job.
His campaign focused heavily on attacking and questioning the loyalty of the country's Arab minority - a strategy that drew accusations of racism and incitement from Arab leaders.
"In the coming days we will convene negotiations to assemble a strong Zionist government and to prevent a dangerous anti-Zionist government," he said.
He claimed that Arab parties "negate the existence of Israel as a Jewish and democratic state" and "glorify bloodthirsty murderers."
The partial results released on Wednesday by the Central Election Commission were based on 35 percent of the vote counted.
The three Israeli TV channels reported the same outcome, based on more than 90 percent of the vote counted, but did not explain the discrepancy with the commission's percentage.
Final results are expected Wednesday and could still swing in Mr Netanyahu's favour.
According to the partial results, the parties of Mr Gantz and Mr Netanyahu received 32 seats each in the 120-member parliament.
Likud with its natural allies of religious and ultra-nationalist parties mustered 56 seats - or five short of the needed majority.
This means both Likud and Blue and White will have difficulty setting up a governing coalition without the support of Avigdor Lieberman's Yisrael Beitenu party.
That put Mr Lieberman, a former protege of Mr Netanyahu's who has become one of the prime minister's fiercest rivals, in the position of kingmaker.
Arab parties, which have never before sat in an Israeli government, also finished strong, and exit polls predicted they would form the third-largest party in parliament.
Addressing his supporters late Tuesday, a jubilant Mr Lieberman said he saw only "one option": a broad, secular coalition with both Blue and White and Likud.
"We've always said that a unity government is only possible in emergency situations. And I tell you and I tell every citizen today watching us on television: the situation, both security-wise and economically, are emergency situations," he said. "The country, therefore, requires a broad government."
- Read more
Early Wednesday, Mr Gantz told a cheering rally of supporters that while it was too soon to declare victory, he had begun speaking to potential partners and hoped to form a unity government.
"Starting tonight we will work to form a broad unity government that will express the will of the people," he said.
Attention will now focus on Israel's president, Reuven Rivlin, who is to choose the candidate he believes has the best chance of forming a stable coalition.
Mr Rivlin is to consult with all parties in the coming days before making his decision.