The shadow foreign secretary said: "He is in the end a liar; he's callous - he doesn't care. He plays at politics; he plays with people's lives.
"He needs to be held to account."
Ms Thornberry, who was Mr Johnson's opposition when he was foreign secretary, added that he has a "woman problem".
"He certainly has a problem with me," she said.
As the crowd began to react to her comments, she continued: "And I want to take him on again as leader of the opposition.
"Because that prime minister needs to be exposed."
She added that he was "the single worst foreign secretary the country has ever had" and that he "didn't like it" when she challenged him.
Ms Thornberry is taking part in the hustings alongside the other four candidates vying to replace Jeremy Corbyn - Jess Phillips, Lisa Nandy, Keir Starmer and Rebecca Long-Bailey.