The former education secretary said the capital has benefited from diversity in its population, helping to drive up standards in education.
He added that in many cases, some of the parents with the highest expectations for their children are those who have come to London as refugees, citing those from Somalia and Kosovo as examples.
But Mr Gove also warned that there are costs to migration, with class sizes forced to grow in some cases and families missing out on places at favoured schools.
Speaking at the Global Education and Skills Forum in Dubai, Mr Gove said: "There's lots of evidence that London having become more diverse has contributed to educational standards rising in London.
"I'm not saying that's the only cause, far from it, but I do think it is the case, and I know this from my experiences as a parent with two children in London state schools, London state comprehensives.
"Many of the children, or whose parents more to the point, have come here in recent years, have extraordinarily high expectations of the state system and so some of the most demanding parents, as well as the most involved parents, are those who have come from... they might be refugees from Somalia or Kosovo.
"They want their children to succeed in Britain, as British citizens, and so they have very, very high expectations of the school, both academically and also as institutions that give people a sense of Britishness."
But he added: "It's undeniably the case that there that there are other parts of the United Kingdom where educational performance is lower and which are often areas where some of the same pressures haven't applied.
"So I'm not saying there's an automatic equation between new people arriving and higher expectations and all of that being wonderful, but I do think it's the case that as we develop a balanced approach towards migration we need to recognise that it has contributed in some cases to a drive towards higher standards.
"It has also contributed, in other cases, to pressure on services.
"So, there have been rising class sizes and difficulty for some people getting their children into the school they wanted as new arrivals from other countries have made that more difficult."
Many parts of England, particular major cities, are facing a squeeze on school places.
Mr Gove, who campaigned to leave the EU during the Brexit referendum, said that while migration can be "a really good thing for a country's economy", if a nation changes too quickly, a strain can be placed on public services and this poses questions for governments who have to balance all sides.
Additional reporting by Press Association