After a stalker was arrested on Monday for lurking around Rihanna's Pacific Palisades home while she has been away in the U.K. on her Diamonds World Tour, a mob of her neighbors are now officially fed up with all the trespassing and paparazzi drama.

The people who reside in the community are used to having peace and quiet. Once the "Stay" singer decided to join in, the unruly fans and photographers were sure to follow.

"We have had it," said nearby resident Marian Mandel. "We're quite upset about the publicity. It was a private street and we want to keep it that way. Everyone is taking care of each house and we are protecting our rights up here."

Monday's incident wasn't the first time a deranged fan attempted to make an unwelcomed visit to her $12 million home. Back in March, Rihanna had to get a three year restraining order against a man by the name of Steveland Barrow who accidently broke into a neighbors home thinking it was owned by @badgalriri.

A neighbor who remained anonymous feels that those who live the closest to her are the most irritated by the situation.

"The neighbors closest to Rihanna are really offended by this," said the neighbor. "It affects our entire street and it's no longer private because of her. Trespassers come here just to look for her."

Mark Green who lives right next door to Rihanna says the community is being aggressive when brainstorming on the next step they can take to keep where the live private and safe.

"We've been getting emails and alerts about what happened," said Green. "Each has spoken to different security companies and we're formulating if we go in as a group or contract separately."

Green doesn't put all the blame on Rihanna though. Due to her being absent most of the time because of her busy schedule, she can't help who trespasses.

"As long as people at least keep getting arrested, it's all good," he said. "Other neighbors are pretty upset, but I'm okay. It's a process. I can count on one hand the number of times I've had to warn people away. She has no control over what crazy people do."

Mandel is predicting and possibly hoping that the singer will one day throw in the towel.

"She'll probably move out," she said. "She'll get tired of this."