Mr Trump has chosen not to release his tax returns since his 2016 campaign, claiming an audit by the Internal Revenue Service prevented him from making the financial information public.
In an interview with Fox News on Sunday, however, Mr Giuliani revealed that "there should be some finality in tax returns".
"All these tax returns have been by and large -- maybe not the last one -- but all of them have been audited, all of them have either been passed on or settled," he said.
"There should be some finality in tax returns. We get audited, we make a deal, we pay the government, you don't come after me forever for that."
By comparison, the White House's position is that Mr Trump was not releasing the financial documents due to the continuing audit.
When asked why Americans shouldn't believe the president is trying to hide something, press secretary Kayleigh McEnany said during a media briefing that the president's taxes remained under audit and he would release them "when they're no longer under audit".
Mr Giuliani suggested in his Fox News interview that the president's last tax return remained under audit, and Ms McEnany said last week that the Supreme Court did not rule against Mr Trump in its decision to send a subpoena of records back to a lower court.
"They even recognised that the president has an ample arsenal of arguments that he can make," Ms McEnany said.
Mr Giuliani said the argument could be made that there was "no particularized need" for the tax returns as there was no investigation of any particularized crime.
"They have no reason to believe that there's anything wrong with his tax returns," he said.
"We can argue that they have no need for it, this is a fishing expedition, I think we fight them right down to the wire," he continued.