Labour's Andy Sawford has been elected MP for Corby and East Northampton, winning a key marginal seat from the Conservatives.
Mr Sawford secured a majority of nearly 8,000 in seizing the Commons seat from the Tories, comfortably defeating rival Christine Emmett.
The Liberal Democrats humiliatingly lost their deposit despite demanding a last-ditch recount.
It is the first time in 15 years Labour has taken a Tory seat in a by-election, giving a boost to Labour leader Ed Miliband.
The by-election in the Northamptonshire seat was forced by the resignation of Conservative MP Louise Mensch to spend more time with her family in the US.
Turnout was just under 45%, in stark contrast to the police and crime commissioner elections, which also took place on Thursday and saw as few as 10% of voters bothering to cast a ballot in some areas.
Victorious Mr Sawford suggested it had set the party on track for a successful general election in 2015, declaring: "The road to Downing Street runs through Corby."
He said the result was "a damning verdict on (David) Cameron's betrayal of the British people".
But the Prime Minister dismissed the outcome as "a classic mid-term result".
Following the result, Mrs Mensch said on her Twitter page: "Given circumstances, v strong performance indeed by Christine Emmett."
She added: "Didn't expect Lib Dems to lose deposit though. Meanwhile, wish @AndySawford the very best in representing #Corby."
Before the result was declared, Mrs Mensch said she would accept responsibility for an election loss.
Again on Twitter, she said: "Election result will not be a verdict on either Christine, or the Conservatives, but only on the decision I took to step down mid-term."
She added: "Though I had to resign to unite our family, there was large & entirely understandable local anger. I wish I had been able to see out my term."
Conservative Party co-chairman Grant Shapps took a swipe at Mrs Mensch, saying he believed MPs who stood for election "should stay there".
He told Sky News: "I saw a tweet from her saying that actually she understands why people would be miffed.
"I have been there and I've heard what people say on the doorstep - MPs should stay the course. I think it would be very helpful to stand for the full parliament."
The result came as Mr Cameron struggled to defend the Government's handling of the PCC elections.
The Prime Minister insisted the victorious candidates still had a mandate, arguing that turnout was always going to be low for new posts.
But the Electoral Commission announced that it will carry out an inquiry into the lack of public engagement, saying it had disagreed with "a number of decisions" taken by the coalition.