Mr Heaton-Harris said the restoration of Stormont’s ministerial executive is his priority in his new role.
The DUP is blocking the functioning of the power-sharing institutions in Belfast as part of the party’s protest against Brexit’s Northern Ireland Protocol.
Political opponents have heavily criticised the DUP boycott, insisting it is hampering efforts to support families struggling amid the cost-of-living crisis.
The Government has vowed to secure changes to the protocol, either by way of a negotiated compromise with the EU or through controversial domestic legislation that would empower ministers to scrap the arrangements without the approval of Brussels.
There are concerns the latter approach could spark a trade war with the EU.
Speaking briefly to reporters on Thursday, Mr Heaton-Harris said: “I am looking forward to delivering enough pressure so we can get the Executive up and running, solve the problems of the protocol, and some of the more useful domestic things I can do to help people here in their everyday lives.”
Sir Jeffrey said he is under no pressure to reengage with Stormont and insisted his party will not do so until action is taken to resolve the issues created by the protocol.
“I don’t feel under pressure at all because I am moving forward on the strength of the mandate I was given by the people of Northern Ireland,” he said.
Sir Jeffrey said he believes Mr Heaton-Harris and new Northern Ireland Office (NIO) minister Steve Baker “understand the nature of the difficulties we face” in respect of the protocol.
Other parties in Northern Ireland have questioned the appointment of both men to the NIO, given their respective associations with the hardline Eurosceptic ERG.
Sir Jeffrey urged his opponents not to get “hung up on personalities”.
Referencing Sinn Fein, he said: “There’s no point bleating from the sidelines or complaining about the people who are at the other side of the table.
“What we need is grown-up politics and it’s time Sinn Fein grew up when it comes to respecting the UK Government and the mandate that we have to deliver a solution on the protocol.”
Sir Jeffrey did not discount a negotiated settlement emerging from renewed talks between the UK and EU.
“I think there is the prospect of renewed negotiations,” he said.
“I think that will require a change of stance by the European Union, that they need to recognise that if we are to arrive at a solution, that requires them to accept and respect the integrity of the United Kingdom, its internal market and Northern Ireland’s place within it.”
Earlier, Sinn Fein vice president Michelle O’Neill told Mr Heaton-Harris that people in the region “desperately” need an executive.
The pair held talks in Dungannon, Co Tyrone, on Thursday morning.
Ms O’Neill said she also stressed to the new minister that the post-Brexit protocol is “here to stay”.
Sinn Fein’s Stormont leader welcomed the Government minister’s comments on his determination to see the executive restored, but said the “proof of the pudding will be in the eating”.
She added: “Time will tell if there is a change of tact from this new Tory leadership.
“I took the opportunity to make it very clear to him what my priorities are. Number one, getting an executive up and running again.
“The public desperately demand that; they need that to help them through this cost-of-living nightmare that people are facing.
“I have said to him that four months post-election, the outcome of that election has yet to be respected and people are still being denied access to a locally-elected executive that could help them through this period.
“I have also made it very clear that people are struggling right now. People are really worried about how they are going to heat their homes, how they are going to feed their kids.
“They need our help and I want to be in an executive, working with others to get support directly out to people.
“We made the case strongly that this situation is no longer tolerable. They need to stop pandering to the DUP.
“The DUP need to get round the executive table with the rest of us and support people.
“The other issue I pressed home to him is that the protocol is here to stay.
“We need an agreed way forward. We do not need more unilateral action. What we need is a negotiated way forward within the framework of the protocol.”
Sir Jeffrey, who met Mr Heaton-Harris later in Belfast, insisted that cost-of-living support could be delivered to people in Northern Ireland despite the ongoing absence of an executive.
“Let me be clear: my party will work day and night to ensure – at Westminster and through our ministers at Stormont – that people get the help they need, households and businesses alike in Northern Ireland get the support that they require to help them with rising energy costs and to meet the cost of living at this time,” he said.
After the meetings with the DUP and Sinn Fein, Mr Heaton Harris said he had reiterated the importance of restoring stable government at Stormont.
He said he had also listened to concerns around rising energy prices and said he emphasised the UK Government’s commitment to taking action to support people and businesses in Northern Ireland with their bills.
“We currently face a range of challenging issues, including finding a solution to the Northern Ireland Protocol that works for the people and businesses here, and the rising cost of living,” he said.
“Today, the Prime Minister has announced a significant package of support to help households, businesses, charities and public sector organisations across the UK with their energy bills, and I have already been speaking to my Cabinet colleagues to ensure we can see support delivered in Northern Ireland as soon as possible.”
The Secretary of State has pledged to meet with the leaders of the other main parties in Northern Ireland in the coming days.