Bob Thomson, a 72-year-old former plasterer, suffered a swollen jaw and two chipped teeth during the unprovoked attack in Rotherham.
Mr Thomson, who voted Remain in the referendum but believes the UK should honour the result, told The Independent: “To be honest, I think it was the Brexit thing that did it... He was a Brexiter. He was very anti-Labour, his aggression was against Labour.”
But the victim said he was determined to continue campaigning for a better society and expects to be back out doorsteps soon.
The local Rother Valley constituency, which has had a Labour MP since it was created in 1918, voted nearly 67 per cent for Leave in 2016.
Mr Thomson was canvassing with other activists in his local village of Hellaby, Rotherham, on Sunday afternoon.
After getting no answer at a semi-detached house in Parkstone Crescent, he posted a leaflet through the door and started walking away – only for the resident to call him back.
But it quickly became clear that the man was not interested in Labour's policies on NHS or pledge on women’s pensions, Mr Thomson said.
And when the activist brought up Brexit, the resident and his wife kept interrupting him before he could explain his views, he added.
“He shouted me down... they were taking the Mickey,” said Mr Thomson. “He did shout something but I wasn’t taking any notice.
“I said, ‘Look this is going nowhere, I’m wasting my time, I will get off’.
“I got to the bottom of the drive and he screwed the leaflet and threw it at me. Then he stood in front of me having a go at me.
“I was looking him in the eye and then suddenly he gave me a right cow tailer [punch]. By this time he was in a bit of a rage.”
Mr Thomson, who is generally in good health despite previously suffering a stroke, said he believed the attack might have continued if the man’s son had not come running out of the house to intervene.
The 6ft 2in pensioner initially shrugged off the blow and walked off to join to his fellow canvassers.
However, after consulting with his family, he decided to call police before driving to hospital, where doctors reassured him he had not suffered a broken jaw.
“I couldn’t close my mouth properly and I was having trouble swallowing,” he said. “They said it was soft tissue damage. I also went to the dentist for two chipped teeth”.
Mr Thomson said that the incident had left his wife “frightened to death”, adding: The whole family are traumatised by what happened.”
He said he could not remember what the Brexit supporter was shouting before the attack, but added: “It was about me... he was telling me he didn’t like me, but I wasn’t taking any notice.”
But the incident has not dimmed Mr Thomson’s passion for campaigning. “I don’t do it for the recognition, I feel that we need to get our society sorted out.”
A 51-year-old man was arrested on suspicion of causing grievous bodily harm in relation to the incident and has since been released on bail.