Prof Anthony Harnden, the deputy chairman of the joint committee on vaccination and immunisation (JCVI), said there "isn't a strong scientific argument" to immunise teachers, telling MPs on the science and technology committee that workers in meat processing factories would have a better case to jump the queue.
Prof Harnden said deviating from the age-based approach at all could lead to "harm" by slowing down the overall number of doses administered each day.
The committee has submitted its recommendations to ministers, who are expected to make a final decision on the second stage of the rollout in the coming days.
All adults aged 50 and above, as well as patients of all ages over 16 with underlying health conditions, are expected to have been offered a first vaccine dose by the end of April.
Watch: Williamson statement - Covid testing in schools
In recent weeks, ministers have faced repeated calls from teaching unions to prioritise teachers for vaccines before reopening schools. Police representatives have also called for officers to get jabs.
Prof Harnden, a professor of primary care at Oxford University, said: "Actually, when you look at the Office of National Statistics data, particularly that school survey of infection data, it doesn't suggest that teachers are any more at risk of acquiring infections or coronavirus than any other member of the population.
"So in terms of infection, and in terms of disease, there isn't a strong scientific argument to immunise teachers outside those aged 50 or those with underlying health conditions."
Prof Harnden emphasised that any decision to prioritise particular groups of workers would be for politicians, and said the JCVI would be "steering our advice based on science".
He added: "I would say that one of the key reasons that this programme has been so successful is because it has been simple, it's been deliverable, it's been rolled out very quickly and people understand it.
"If you start picking out certain groups, it will make it more complicated. The risk of doing that is slowing the programme down, and then it may be that some people will be exposed to virus and actually some harm that wouldn't have been otherwise."
As of Tuesday night, more than 18,242,873 first doses, plus 669,105 second doses, had been administered across the UK.
Ministers promised last week that the pace of the vaccine rollout would double between then and the end of April as patients increasingly return for a second dose.
Sir Simon Stevens, the head of the NHS, announced the start of a "second sprint" in which the health service will aim to vaccinate all remaining adults over 50 plus others with underlying conditions, or groups five to nine on the JCVI list. The rest of the adult population will form phase two of the rollout.
Prof Harnden's comments came as teachers at special schools were told they would have to wait for a Covid jab despite office-based council employees having already being offered the vaccine.
The row emerged after staff at special schools in North Tyneside demanded to be vaccinated on the basis that working with children with special needs puts them at added risk. It is a picture that has been repeated elsewhere in the UK, with some special school staff being vaccinated and others not.
They have asked to be prioritised because some pupils cannot wear masks, while others struggle to maintain social distance and some forget to catch their coughs and sneezes.
On Wednesday night, it was reported that the JCVI has recommended that prisoners be vaccinated en masse as part of the phase two rollout. Government sources said this was not a formal recommendation, rather "a practical acceptance that local areas can have this flexibility if they need it".
Watch: How England will leave lockdown