David Cameron will promise to remove "absurd barriers" to mixed-race adoptions later when he unveils plans to speed up the system.
The Prime Minister said there was "no more urgent task" for the Government and vowed to legislate to force councils to work faster.
The current average time between a child going into care and being placed with adoptive parents is one year and nine months.
But the changes will see councils forced to approve adopters more quickly and make sure the question of ethnicity comes second to the speed of finding a suitable home.
Town hall social workers would also be forced to turn to the existing national adoption register if they fail to place children locally within three months.
Mr Cameron said it was "shocking" that black children had to wait an average of twice as long to be adopted and vowed the Government would tackle the "absurd barriers" to mixed race adoption.
"This government is going to tear down the barriers that stop good, caring potential adoptive parents from giving a home to children who so desperately need one," he said.
"We will tackle the absurd barriers to mixed-race adoption which trap many non-white children in care. We will make sure that local authorities who let children down make faster use of the National Adoption Register.
"And we will remove obstacles to make sure potential adoptive parents can be foster carers too, so that it's no longer too hard for children to be placed with them while final decisions are made.
"Together, these are vital steps towards a system that is fairer, faster, and puts children and parents first."
He added: "Every child deserves the love of a stable family - and that's why I have made sorting out and speeding up adoption in this country a priority.
"There's no more urgent task for government than this. Young lives are being wasted while the process takes its toll."
Children's minister Tim Loughton said: "We need to change legislation to make sure that reducing delay is the primary concern of social workers and adoption managers across the country. A year is such a significant proportion of a young child's life. We can't afford to waste time."
The current law states that a child's welfare should always be the "paramount consideration" but also requires that religion, race and culture be taken into account.
Revised statutory guidance was issued last year in an attempt to address the delays faced by ethnic minority children but ministers believe it has not had enough effect.
Downing Street said primary legislation would now be brought forward to make it clear "ethnic matching" should not automatically be the overriding factor.
Secondary legislation is also expected to be used to make it easier for children to be fostered with approved adopters who could then take them in permanently.
Martin Narey, the Government's adviser on adoption who previously headed children's charity Barnardo's , said he was "delighted at the urgency" being shown by ministers.
"The announcement today will secure the earlier and more successful adoption of many thousands of children whose lives will be transformed.
"Local authorities will be asked to make concurrent planning more widespread so that children are fostered by their prospective adoptive parents as long as it is in the child's best interests."
The moves were also welcomed by the NSPCC but it warned against making speed the prime focus.
Chief executive officer Andrew Flanagan said: "The Prime Minister is right to address the impact that delays within the care system can have on children but this work must be driven by their best interests, not timescales.
"The most important outcome for any child is to have a safe, secure and loving home environment, whether that is through fostering, adoption or being reunited with their birth family or other permanent option."
Full details of the shake-up are due to be published by Education Secretary Michael Gove next week, who was himself adopted.
The Government has said it would introduce the changes "as soon as possible" but officials would give no commitment to the changes being made before 2015.