Boris Johnson says he will not hesitate to move countries off the green travel list but could see "nothing in the data" to mean the 21 June easing of lockdown measures cannot go ahead.
The prime minister warned: "I want you to know we will have no hesitation in moving countries from the green list to the amber list to the red list, if we have to do so."
Mr Johnson, speaking ahead of an expected review of the travel traffic light system on Thursday, said the government is "going to try to allow people to travel".
"But we've got to be cautious and we've got to continue to put countries on the red list, on the amber list, when that is necessary," he said.
Currently, Portugal and Iceland are the only viable tourist destinations on the green list, meaning visitors returning to the UK have to take one PCR test but do not have to quarantine unlike travellers returning from countries on the amber and red lists.
Giving nothing away, Mr Johnson said people will have to "wait and see what the Joint Biosecurity Centre say" as they review the list.
There are hopes some of the Greek islands will be added to the green list, as well as Grenada, Antigua, Malta and Finland - but there are also fears some southeast Asian countries including Malaysia, Thailand, Vietnam, Cambodia and Indonesia could be added to the red list, meaning 10 days in a quarantine hotel.
The prime minister has also said he could see "nothing in the data" to suggest stage four of the lockdown roadmap cannot go ahead on 21 June.
The fourth and final stage of the prime minister's roadmap for easing coronavirus rules - dubbed "freedom day" and when the government aims to remove all legal limits on social contact - is scheduled to take place in less than three weeks.
But, in recent days, Mr Johnson has faced growing calls from scientists to delay the full lifting of restrictions due to the spread of the Indian (Delta) variant of coronavirus, which is believed to be more transmissible, within the UK.
Speaking in Downing Street on Wednesday, the prime minister appeared positive about the prospects of his roadmap remaining on track for 21 June.
Yet he stressed a need to be "cautious" due to "ambiguous" data on how successful the UK's vaccine programme would be in combatting a new surge in infections.
"I can see nothing in the data at the moment that means we can't go ahead with step four of the opening on 21 June," he said.
"But we've got to be so cautious because there's no question the ONS (Office for National Statistics) data of infection rates is showing an increase.
"We always knew that was going to happen. Don't forget we always said the unlocking steps that we've taken would lead to increases in infection.
"What we need to work out is to what extent the vaccination programme has protected enough of us - particularly the elderly and vulnerable - against a new surge.
"There, I'm afraid, the data is just still ambiguous and so every day we're having long sessions where we're interrogating all the data, we're looking at all the various models.
"And the best the scientists can say at the moment, in their guidance to us, is that we just need to give it a little bit longer."
Mr Johnson acknowledged the wait for a clear answer would be "frustrating" for people ahead of stage four of his roadmap, which is set to see nightclubs reopen and larger events allowed once more.