David Buik said staff had to get back to their desks to get society “firing on all cylinders”.
He made the comments as Boris Johnson was expected to scrap guidance telling people to “work from home if you can”.
However, ministers are not expected to launch a campaign to encourage staff back and it will be left to employers and staff to decide.
Mr Buik described the City as “like a morgue” and said infrastructure will “crumble” if people do not start going back into the office.
He told the Standard: “If all the sandwich shops close it’s like a domino effect. All the restaurants close, all the bars close, the number of people go out of work. All of a sudden you’re looking at three million unemployed.”
He also told LBC: “Working from home works quite well three days a week or two days a week, but the fact remains is that the infrastructure of society will crumble.”
However, he said he was pleased some business leaders were encouraging people back into the office.
Last week the Standard reported that Goldman Sachs described offices including its main one in London as the “centre of gravity”.
Richard Gnodde, the bank’s international head, said: “We evolve in a creative industry, which presupposes teams working together. Young bankers learn the job by observing senior ones.”
A lack of a campaign to encourage people back to the office will be a big disappointment to London business leaders who see a mass return to the office as critical for the capital’s recovery. However, ministers see it as impossible to set “one size fits all” rules.
Meanwhile, civil servants are not being asked to go back to their desks in Whitehall this summer.
Some ministers have privately expressed annoyance that their officials are only expected to be asked to work in the office four days a month from September, rising to eight days a month next year.