Judge Judy Says Former Neighbor Justin Bieber Used to Avoid Her Because He's 'Scared to Death of Me'

It's safe to say Justin Bieber probably isn't keeping former neighbor Judge Judy Sheindlin on his holiday greetings list.

The TV judge, who used to live beside the "Ghost" singer, claimed he used to be terrified of her after she spoke publicly about his teen years. Judge Judy, 80, explained how she knew Bieber, now 28, was fearful of her.

"He's scared to death of me," she told Access Hollywood. "There was a period of time before he grew up — when he was foolish, and doing foolish things."

RELATED: Judge Judy: Her New Show — and the Amazing Story Behind Her $460 Million Fortune

Dimitrios Kambouris/Getty Justin Bieber

With a laugh, Judge Judy  recounted that Bieber allegedly had his security alert him of the Judge's time outside — just so he wouldn't accidentally cross her path.

"I must have said something about it, and then, I understood that he was paying the front door people to let them know when I was there, coming and going, so he wouldn't have to bump into me," she said.

RELATED: Justin Bieber and Wife Hailey Welcome a New Puppy into the Family Named Piggy Lou: 'Baby Sister'

Judge Judy did publicly speak about Bieber after he was arrested in 2014 for a DUI. At the time, she told CBS Los Angeles how he may be taking his celebrity status for granted.

David Crotty/Patrick McMullan via Getty, Johnny Nunez/Getty Images for The Recording Academy

"Being a celebrity is a gift," she told the station. "You could either treat it reverently or you could make a fool out of yourself. And he's doing a very good job of making a fool out of himself."

She added, "I think it's sad. And nobody's going to remember that he was a marginal singer.  But they're going to remember a young kid who had a chance to have it all and who is blowing it by acting like a fool."

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

Now, with the DUI eight years behind him, Bieber has settled down with wife Hailey Baldwin Bieber.