Asked by reporters if travelling more than 250 miles from his London home during lockdown "looked good", Boris Johnson's chief aide said: “Who cares about good looks? It’s a question of doing the right thing."
He then added: "It’s not about what you guys think.”
Opposition MPs have accused Number 10 of a “cover-up” and called for Mr Cummings to resign after it emerged that he had travelled to Durham, days after travelling restrictions were enforced as a result of the coronavirus outbreak.
But Downing Street has defended his actions , insisting he "acted in line with guidelines" and that his journey was "essential" because he needed childcare.
Downing Street had previously confirmed that Mr Cummings had started displaying coronavirus symptoms “over the weekend” of March 28 and 29.
On April 5, Mr Johnson was admitted to hospital with coronavirus symptoms and later required treatment in intensive care.
An unnamed neighbour told the Mirror and the Guardian, who first broke the story, that on the same day Mr Cummings was seen in the garden of his parents’ home in Durham as Abba’s Dancing Queen was playing loudly.
“I got the shock of my life, as I looked over to the gates and saw him,” they said. "I recognised Dominic Cummings – he’s a very distinctive figure.”
Former Conservative MP David Lidington, who was de facto deputy PM under Theresa May, was among those saying the news raised serious questions.
He told BBC Newsnight: “There’s clearly serious questions that Number 10 are going to have to address, not least because the readiness of members of the public to follow government guidance more generally is going to be affected by this sort of story.”
Piers Morgan joined those criticising the Government's response as he vowed to drive to see his parents if Boris Johnson does not fire Dominic Cummings today.
But Minister for the Cabinet Office Michael Gove, who used to employ Mr Cummings, spoke out on his behalf, tweeting: “Caring for your wife and child is not a crime.”
Caring for your wife and child is not a crime https://t.co/YCXWhKTq28— Michael Gove (@michaelgove)May 23, 2020
Fellow senior Cabinet colleagues, Foreign Secretary Dominic Raab and Chancellor Rishi Sunak, also weighed in to defend the aide.
Mr Sunak tweeted: “Taking care of your wife and young child is justifiable and reasonable, trying to score political points over it isn’t.”
The news about Mr Cummings comes as authorities managing beaches and beauty spots are bracing for a big influx of visitors over the bank holiday weekend, expected to put social distancing rules under strain.
Authorities in many tourist hot spots, including Brighton, Hastings and the Isle of Wight, are urging people to stay away.