The pair were giving evidence after it was revealed they met Russia's ambassador to the UK three times in 2015 and 2016 - rather than the single “six-hour boozy lunch” Mr Banks had previously acknowledged.
During the session it was revealed that Mr Banks handed the Russian ambassador Alexander Yakovenko a telephone number for Donald Trump’s team after his election win.
"We've got places to be, I'm afraid" the former UKIP donor told stunned MPs.
"I really have to insist. I was told a certain time and we've got a lunch appointment that I don't want to be late for".
Pushing back the chair of the committee Damian Collins asked Mr Banks "Could you just give us five minutes of your time?"
His request was met with the reply: "No... the word is no".