There are "quite significant concerns" about employment practices within clothing factories in freshly locked-down Leicester, the health secretary has told Sky News.
It follows allegations that clothing factories in Leicester, many of whom supply major online retailers, have risked spreading COVID-19 by failing to implement additional hygiene or social distancing measures.
There are also claims that some factories told staff to continue to go into work despite being ill.
In addition, specialist National Crime Agency officers and Leicestershire Police have visited a number of businesses in recent days to investigate allegations of modern slavery and human trafficking.
Leicester has been forced to re-enter a strict lockdown following a spike in the number of local COVID-19 cases.
Mr Hancock, appearing on Sky News' Sophy Ridge On Sunday show, admitted the government is "very worried" about the number of coronavirus infections in the East Midlands city.
"They are clearly very high, three times higher than the next highest city in terms of the testing results," he said.
"We put more testing in but that doesn't account for the increase, the increase is over and above that which would have been accounted for by the increase in testing.
"We have seen outbreaks in food factories and in clothing factories; there are some quite significant concerns about some of the employment practices in some of the clothing factories in Leicester.
"They are important problems to deal with but the number one problem that we've got to deal with is getting this virus under control."
The health secretary, who was speaking after what was dubbed "Super Saturday", the biggest easing of lockdown measures yet across the rest of the country, said a "huge amount of extra resource" has been deployed to Leicester.
Asked whether the government had been strong enough in taking action against employers who ignore coronavirus guidance, Mr Hancock replied: "That guidance is statutory guidance, it's backed up by health and safety legislation and there are very significant fines that can be levied if it's broken.
"We also have the authority to be able to shut down a business if it doesn't follow that guidance."
Pressed on whether the government is looking at closing down businesses, Mr Hancock said: "Yes, and that's happened, absolutely.
"Normally that's happened in consultation with and working with the employer, with the business in question, but we do have that power, absolutely.
"We're not just asking nicely, we are very clear to businesses that these are their responsibilities."
But the health secretary stressed there are "many steps that you can take before you fully close a factory".
The Health and Safety Executive recently wrote to the House of Commons work and pensions committee to reveal it has issued no prohibition notices - which would require an immediate cessation of activity - since the COVID-19 pandemic began.
Instead, it has issued two improvement notices, which give businesses at least 21 days to address a risk identified by an HSE inspector.
The HSE had previously incorrectly told MPs it had issued one prohibition notice to a business since the start of the coronavirus crisis.
According to a Sunday Times investigation, one factory in Leicester was operating last week during the localised coronavirus lockdown without additional hygiene or social distancing measures in place.
The newspaper also claimed one of its undercover reporters was told to expect £3.50 an hour in pay, despite it being less than half the minimum wage.