The Government has come under fire from victims of crime for failing to deport hundreds of foreign criminals.
A damning report by the National Audit Office (NAO) revealed 760 foreign national offenders awaiting deportation have absconded and 395 of those have been missing for more than four years.
Some 58 of these offenders are described as "high harm" and present a serious danger to people or property.
David Cameron admitted "the buck stops with me" during Prime Minister's Questions , but he also appealed for support from the other parties over what he called the "obstacles" of human rights legislation.
"We've deported 22,000 foreign national offenders since I've become Prime Minister," he said.
"The report is very clear that since 2013 for the first time we've got a proper cross-government strategy to deal with this."
Home Secretary Theresa May told MPs she is acting to end the abuse of the legal process.
She said: "As the report makes clear, this is a problem that has beset successive governments. It falls to this government to tackle the problems of the past.
"Quite simply, the Home Office did not prioritise the removal of foreign national offenders before 2005."
The report reveals the case of a foreign national whose UK visa had expired, yet authorities took no action over 14 years to remove him.
Even when the Government first began extradition proceedings in 2007 when he was convicted of a string of sex offences, he launched a series of appeals that delayed proceedings for another seven years.
It is estimated that public bodies spent £850m in 2013/14 managing and removing foreign national offenders, working out at around £70,000 per offender.
Meanwhile, the number of foreign prisoners has risen 4% from 10,231 to 10,649 since 2006, the NAO said.
The number of those removed has fallen to 5,097 from a peak of 5,613 in 2008/09.
The time it takes to deport an overseas criminal has been revealed to be 319 days.
This comes despite a 10-fold increase in the number of Home Office staff working on foreign national offenders (FNOs), from 100 to more than 900.
Amyas Morse, of the NAO, said: "It is no easy matter to manage foreign national offenders in the UK and to deport those who have completed their sentences.
"However, too little progress has been made, despite the increased resources and effort devoted to this problem."
Conservative MP Philip Hollobone has long raised concerns about the number of foreigners in UK prisons and failures to deport them.
He said: "Most people will be staggered that despite increasing its staffing for deportations from 100 to 900, the Home Office is not actually deporting any more FNOs than it was before.
"My view is that if you are a foreign national who commits a crime in the UK, you should be caught, convicted and sentenced with your sentence served back in your own country at the expense of your fellow nationals."