The shadow treasury minister said many black people woke up with a "sense of dread" the day after the referendum.
He told Sky News' Sophy Ridge On Sunday programme: "I think part of the Brexit campaign, and part of the undertone of Brexit, from some politicians, Nigel Farage and others, had racism at its core and its heart.
"They used it as a mechanism to divide our communities, to divide our country."
He added: "How many people of colour woke up on the day after the referendum with a sense of dread because of what had happened?
"Ultimately our country had chosen to listen to Boris Johnson, someone who has a track record of racist commentary, of giving credence to racism."
Mr Lewis also suggested the Duchess of Sussex has been the victim of "structural racism" in the media.
"We can see it with Meghan Markle and the way that she's been treated in the media. We know this is a reality of the 21st century still after 400 years of racism," he said.
Mr Lewis currently has just four nominations from MPs and MEPs, 18 short of the total he needs to secure by 2.30pm on Monday if he is to make it through to the next stage of the contest.
Shadow foreign secretary Emily Thornberry, who only has 10 nominations, has said she is "fairly confident" she will get the numbers needed by the Monday deadline.
"From the conversations I have had this weekend I am fairly confident that, as long as I don't get any slippage I will be fine. I am going to get across the line and then we will move on to the next stage," she told BBC1's The Andrew Marr Show.
Four other contenders - Sir Keir Starmer, Rebecca Long-Bailey, Lisa Nandy and Jess Phillips - have already secured the support they need to go forward.
Ms Long-Bailey, the favourite of the left, dismissed suggestions that she was simply the "continuity Corbyn" candidate in the contest.
"It annoys me when people say that and unfortunately as a woman, it annoys me even more. I'm a person in my own right," she told the Sophy Ridge On Sunday programme.
"I would describe myself as a socialist. I got involved in politics because of my principles and it's my principles that drove me to stand to become leader of the Labour Party."