The education watchdog has criticised schools for not using Government money given to them to help poorer students effectively.
In a report, Ofsted found half of the schools which had received a chunk of £1.25bn in funding from the Government to help poorer students had not allocated the money to specifically help them.
Instead, many schools were absorbing the "pupil premium" into their overall budget.
Sir Michael Wilshaw, Ofsted chief inspector, told Sky News that schools "have got to be accountable for this money" and must be able to demonstrate how it was helping them "to close the gap between the richest and poorest".
But headteachers said it was "impossible for schools to separate out poorer pupils from the rest".
"It won’t work and it's not fair," added Martin Johnson, deputy general secretary of the Association of Teachers and Lecturers.
Sir Michael said finding ways of spending the "pupil premium" to boost the achievements of disadvantaged youngsters was not "rocket science".
Ofsted has been surprised to find that half of schools in a survey reported that the money had made little difference to how they are managed and operated, he said.
Only 10% of schools said it was having a "significant" effect - all of which were in the most deprived areas.
Sir Michael added: "If this money is going to the main school budget and children from poor backgrounds are doing well, we do not have an issue with that and I'm sure that Government won't, that is not a problem as long as they can demonstrate that is happening.
"It will be an issue if it just falls into the main school budget and a school can't show that it is improving the outcomes for poor children."
The money should be put towards teachers who are effective at teaching children from poor backgrounds, he said, and other initiatives such as extra classes.
The pupil premium, a flagship Lib Dem policy in the 2012 general election, was introduced in April last year.
This academic year, schools were allocated a total of £1.25bn for children from low-income families who were eligible for free school meals and those from families with parents in the Armed Forces.
The Department of Education said schools were given freedom over how to use the money, but agreed with Ofsted that schools needed to use it properly.
Newly appointed Schools Minister David Laws said he welcomed the "valuable" report and Government needed to "learn from it".
But Labour's Shadow Schools Minister Kevin Brennan said the findings showed that the pupil premium "is not working in the way it was intended, because it fails to offset the cuts the Tory-led Government has made to the schools budget".
Chris Keates, general secretary of teaching union the NASUWT , said the pupil premium should be ring-fenced with close scrutiny of how it is spent.
"The pupil premium was never, despite claims to the contrary by ministers, 'new' money for schools," she said.
"The fact that it was introduced at a time of savage cuts to the education budget and it was left to the discretion of schools on how to spend it has resulted in the premium being simply swallowed up in schools' budgets."
