The UK Border Agency is being split in two and brought back under the control of the Home Office, Theresa May has announced.
The Home Secretary unveiled the shake-up in the Commons 24 hours after MPs published a damning report about the organisation's record.
Mrs May said the current agency was too big, not transparent enough, had failing IT systems and struggled to operate in the existing legal framework.
It will now become two separate operations, with an immigration and visa service assessing applications and an enforcement arm to trace people breaking the law.
Mrs May said: "The agency has been a troubled organisation since it was formed in 2008 and its performance is not good enough.
"In truth, the agency was not set up to absorb the level of mass immigration that we saw under the last Government.
"This meant the agency has never had the space to modernise its structures and systems and get on top of its workload."
It was a mistake to keep it at arms length from ministers because this "created a closed, secretive and defensive culture", she added.
A new board chaired by the Home Office permanent secretary Mark Sedwill will be formed to oversee every part of the immigration system.
There are also plans to modernise IT across the board to speed up case handling.
And in a move to make it easier to remove illegal immigrants, Mrs May said she intended to bring forward an immigration bill in the next parliament.
A leaked internal memo revealing staff had been told they would "still be doing the same job in the same place with the same colleagues for the same boss" risked undermining the reforms.
It is the second time the Home Secretary has had to shake up the agency in the wake of major criticism about its operations.
Last year, the Border Agency was divided from the Border Force - which handles frontline controls at ports - after it emerged hundreds of thousands of people had entered the UK without being checked against a watch list.
The latest changes follow a scathing report from the Home Affairs select committee which warned on Monday that UKBA's backlog would take 24 years to clear.
The committee also attacked the head of Britain's tax office Lin Homer for her "catastrophic leadership failure" when she ran the agency.
MPs were "astounded" that she was promoted to the £180,000-a-year role of chief executive of HM Revenue & Customs after her performance there.
Shadow home secretary Yvette Cooper admitted the UKBA had suffered with problems for years but also claimed its record had worsened under Mrs May's watch.
"Enforcement has got worse, delays have got worse, 50% fewer people are being refused entry at ports and borders and you said the number of illegal immigrants removed does not keep up with the number here illegally - that's because you're letting rather more of them in," she said.
Ms Cooper highlighted a 34% cut in the UKBA budget and a 20% rise in the backlog of asylum cases in the last three months.
"Your reforms and your cuts is what has gone wrong throughout 2012. Why should we believe the latest round of reforms will do any better?," she asked.
Committee chairman Keith Vaz said after the shake-up: "Ministers are now on the front line.
"Proper accountability and scrutiny of our immigration system must continue, and it will need effective and strong leadership if the Home Office is serious about having a fully functional immigration system."
Lib Dem Julian Huppert added: "Lib Dems have always argued you have to have a proper enforcement agency and exit checks, and that's what the Coalition is doing.
"Building a fairer society means making sure everyone plays by the rules. That means an immigration system that works for those who should be here, and acts against those who shouldn't."